When my guests use "Add to calendar," the event is all day. Why?
We let the host enter "free form" in the Time field (e.g., they can enter "noon to 4 pm", or "7-ish" and don't force them to select an exact time). Because of this, we can't import a time when you add to your calendar. Instead, the event comes in as "all day".
We do this because hosts like the ability to write whatever they want for a "time", and guests know that the event is not a 24-hour affair so they just add or edit the time when they import the event to their calendar.