Short answers, pulled from the story.
The First Order is a fictional military movement and rump state introduced in the 2015 film The Force Awakens. It formed after the fall of the Galactic Empire and aims to destroy the New Republic and rule the galaxy as a military dictatorship.
According to the Aftermath trilogy by Chuck Wendig, Imperial Admiral Rae Sloane and Commandant Brendol Hux are the de facto founders of the First Order. Sloane, after being promoted to Grand Admiral, coined the term First Order.
The First Order was originally led by the mysterious Supreme Leader Snoke, who was secretly created by the resurrected Emperor Palpatine. Kylo Ren became Supreme Leader after killing Snoke in The Last Jedi, before Palpatine's return in The Rise of Skywalker.
According to the novel Star Wars: Bloodline, the movement formally took the name First Order only about six years before The Force Awakens. This followed a split in the New Republic Senate between the Populists and the Centrists.
Starkiller Base is the ice planet Ilum converted by the First Order into a mobile superweapon capable of destroying entire star systems through hyperspace. In The Force Awakens, it is used to destroy Hosnian Prime, the New Republic's capital, before the Resistance destroys the base.
The First Order was destroyed after the Battle of Exegol in The Rise of Skywalker, when Rey defeated Emperor Palpatine and the galaxy rose up against the Order. It fell after fourteen years of existence, ending fifty-four years of Imperial power.