Hatteras is very nice then. Before Memorial Day, the crowds are not bad and all the rental cottages rent at lower "shoulder season" rates. Prices go up around Memorial Day, so you'll likely feel better about your rental rate during the first week than the second. (Ask your rental company if you can get the pre-Memorial Day rate for both weeks. Since you're staying 2 weeks, you might get it.)
Wind and weather should be very nice then. If the wind shifts to a southwest (summertime) pattern, you'll feel comfortable without a wetsuit. If wind is hard from the north, you'll still want at least a shorty wetsuit and maybe a steamer top. Lots of people are riding then in all conditions without wetsuits.
Hatteras is definitely worth the drive at that time of year. You didn't say where your drive starts, so it's hard to compare S. Padre. If the distances are roughly the same, I'd choose Hatteras over S. Padre any day. Hatteras is much better for beginners and more fun for experienced kiters, too. Just my opinion, or course.