When my guests use "Add to calendar," the event is all day. Why?
We let the host enter "free form" in the Time field (e.g., they can enter "noon to 4pm", or "7-ish" and don't force them to select an exact time). Because of this, we can't import a time when you add to your calendar. Instead, the event comes in as "all day".
We do this because hosts like the ability to write whatever they want for a "time", and guests know that the event is not a 24-hour affair so they just add or edit the time when they import the event to their calendar.