Heat oven to oven to 350 degrees.
Start by cubing the butter into small pieces and chopping the fresh rosemary. Set it aside to soften while you prepare the rest of the ingredients.
Pour the crackers into the large mixing bowl and roughly break them in half. This doesn’t need to be perfect by any means! Just quickly break them in half in the bowl.
Butter the 9×13 casserole dish (or something similar)
Layer in 1/3 of the crackers in the prepared baking dish. Keep in mind, that you will add 3 layers of the crackers total so save enough for the extra layers.
Add 1/2 of the oysters over the top.
Randomly place 9 small pieces of cubed butter over the top.
Sprinkle about 1/4 tsp of salt and 1 tsp black pepper and 1 tsp fresh rosemary over the top.
Add another layer of crackers, the rest of the oysters, butter, rosemary, salt, and pepper.
The last layer will be the top, so it will just be crackers, the remaining butter, rosemary, salt, and pepper.
Add the oyster liquid in with the milk and then pour the milk evenly over the top of the crackers.
Cover the dish with aluminum foil and bake for 30 minutes covered and 25 minutes not covered. This dish will bake for about 55 minutes.
The crackers will be golden brown on the top.