I'm not the first one to do this but I have't found any pictures or threads, so I figured I'd share.
I initially bought a whole mess of felt (like $0.25 a sheet) and was going to try to somehow cut them all to fit inside perfectly and line the whole thing. I'm now thinking that I'll compromise and do this in a much simpler way. Below are pictures of the felt I ordered, and initial fitment. I think I'm going to mess with it more tomorrow and then set them in with spray adhesive.
You can't tell from the shots but there is a natural hump in the box at the back. The felt in the back was cut to sit in it. Last shot shows roughly what it will look like when peeking in from above the seat.