Others have suggested turning off bluetooth on the tablet, or else go to tablet bluetooth settings and "forget" the pencil. This seems to halt transmissions that drain the battery even when the pencil is sitting in its case and not in use.
If you unpair the pencil, you need to plug it back into the tablet and re-pair it. It's a pain but beats having to keep the tablet in bluetooth-disabled mode all the time.
Now as for my own experience, I do keep bluetooth off most of the time, yet my pencil keeps draining. I just charged it yesterday and already the charge widget shows <100%. By tomorrow it will be probably at 75%.
In my opinion, the pencil is more of a frill than a necessity, unless maybe you're a professional artist or illustrator.
If you have to have one, though, probably the 2nd gen pencil coupled with a newer ipad seems like a better solution; the pencil magnetically sticks to the tablet, and inductively charges, problem solved.