Why celebrate on the 25th of May instead of a more meaningful date?
Because some time was needed to get the word out, Towel Day was organized two weeks after Douglas Adams' untimely death, on 25 May 2001.
Many other dates had been proposed. For those with an exceptionally long attention span: the 42nd day of the year ("Happy Adams Day"), 42 days after his death ("Second Day of Remembering"), the 11th of March (his birthday, around which from 2003 on a yearly "Douglas Adams Memorial Lecture" is held), the last Friday of every May, the Friday before the 42nd week-end day of the year (occasionally falls on 25 May, as it did the year he died), etc.
Of all proposed dates, the 25th of May turned out to be the one that gained a significant following. And over time it has acquired a certain status as a geek holiday. In 2001 Towel Day started to be celebrated. But in 2006 that day also became Geek Pride Day, because Star Wars was released on 25 May (1977). And in 2008, following Terry Pratchett's diagnosis of Alzheimer's disease, Match It For Pratchett called on fans to wear lilacs on 25 May (the anniversary of the Glorious Revolution of the Twenty-Fifth of May, from Pratchett's Discworld books).
As the universe that Douglas Adams created was full of absurdity and randomness, it may be a fitting choice after all. And if you need an additional reason: if you add the hexadecimal numbers 25 and 5, and convert the result to decimal, you get 42!