Short answers, pulled from the story.
"The Tale of Aragorn and Arwen" is a story within the Appendices of The Lord of the Rings narrating the love between the mortal Man Aragorn and the immortal Elf-maiden Arwen. It covers their first meeting, their betrothal on the hill of Cerin Amroth in Lothlórien, and their marriage in Minas Tirith following the War of the Ring. It also covers their deaths: Aragorn's chosen death at age 210 and Arwen's solitary death in a deserted Lothlórien.
Tolkien gave two main reasons: the main story is told from the hobbits' point of view, and the tale extends the narrative both before and after the main events, making it impossible to weave in. The literary historian William Gray adds a third explanation: the tale does not contribute to the narrative "interlacing" the main text relies on, so it could not be threaded into the story's parallel threads.
The tale was published in 1955 in The Return of the King, as the fifth part of the first section of Appendix A. When publishers proposed dropping the Appendices entirely for the first Swedish edition in 1959-1961, Tolkien refused and insisted on keeping the tale. The 1968 one-volume edition of The Lord of the Rings omitted all appendices except this tale.
The tale presents both Aragorn and Arwen as pagan, with no religious consolation available at death. Arwen is not comforted by Aragorn's suggestion that something lies beyond death, and neither is Gilraen, Aragorn's mother, by a similar remark. The medievalist Tom Shippey identifies the result as the "paradox of a 'fundamentally Catholic' work which never once mentions God," since Tolkien absorbed religion into story and symbolism rather than stating it directly.
The tale deliberately mirrors the "Tale of Beren and Lúthien," set in the First Age of Middle-earth, as a second instance of the same pattern: a mortal man loves an immortal Elf-maiden who chooses mortality to be with him. Tolkien, like Dante in the Inferno, placed a story behind a story, with each pair of characters imitating a heroic precedent. The scholar Elena Capra traces both tales to the medieval poem Sir Orfeo, which itself reworks the classical myth of Orpheus and Eurydice.
Within Tolkien's fictional universe, the tale was composed by Barahir, grandson of Faramir and Éowyn, after Aragorn's death. An abbreviated version was later included in the copy of the Thain's Book made by a scribe named Findegil. The actual composition history was explored by Christopher Tolkien in the 12th volume of The History of Middle-earth, working from unpublished manuscripts and drafts that showed the tale passed through several versions.