Darth Vader muses in the Star Wars novel (pg. 106 in my ©1993 ed.) that the most powerful destructive machine ever constructed had no influence on the Alderaan star system—that it would take a micro-breakdown on that section of map to see the "slight reduction in spatial mass" caused by the disappearance of the planet. "The actions of mankind remained unnoticeable to an uncaring, unimaginably vast universe."
Ground your story's setting in physical reality and focus on the plot as not to require heaps of exposition unless the appearance of the planet itself is intrinsic to the plot.
Given the time period of your story, Planet Organa (bleah... go with Thul to spice things up: who says Alderaanians all had to bow to House Organa?) could be a rouge planet, dwarf planet, or some other astronomical object that did eventually respond to the orbital disturbance.