The door card needs to come off to access the speaker. The plastic speaker grille and storage compartment slots are held by screws from behind the door card.
Door card is easy to remove. Total 6 philip screws to undo. Can be done within 15 minutes.
Lift up your door handle and you should be able to gently pry up the cover behind it. No screws -- it just snaps in. Set it aside.
Just remove the three screws under the armrest to remove the lower cover. Set it aside.
Wind down your window, THEN disconnect the power window and lock switch wiring harness.
Undo another 3 more screws holding the armrest and the entire armrest should come right out. Set it aside.
Starting from the bottom two corners of the doorcard, pull AWAY from the door about 1-2 inches to undo the clips and work your way around the door card. You shouldn't need much strength to undo the clips.
The clips look something like this (side view). I just grabbed a pic off Google, but it should give you an idea.
Look behind the door card to make sure that all the clips have come off, then lift the door card UP. Lifting up takes a bit more effort.
The door card basically hooks on to the door frame at the top where the window glass meets.
That's it!
Here's a pic of my door with the doorcard removed.