The idea is to use a brick-shaped baking pan. These 6-10$ pans are almost exactly the shape and size of a brick, and have diagonal sides designed to facilitate an easy removal of the cake. Get one or two of these, depending on how many kids or adults are in the game.
To make a brick, just throw some snow into the pan, and don’t forget to pack it as tight as you can. Flat-out the top surface with your hand or a knife (hay…why not a chainsaw!!!) and turn-over to remove the brick.
Make as many as you like and use them to build walls. Use an interlocking structure (in which each second brick is up-side-down) to get better stability. If you want a real tight construction, spread a little salt on each row before laying the next one. The salt will melt the top layer of ice, and will make the rows fuse together.