Books

HTML 4.0: No Experience Required
Positively, absolutely the best book for learning HTML 4. Period.
HTML 4 Bible
A terrific reference. It also covers the basics other scripting languages and how they interact with HTML.

Web Resources

Webmonkey
A "How-To" guide for web developers (you!). Awesome tutorials and links to web news.
CNET's Builder.com
A great site loaded with information that any web developer will need.