When my grandfather was growing up his father would stuff crushed cloves directly into his cavities to relieve pain. He swears by it nowadays, so I definitely believe clove oil would help.

Thank god dentistry has come as far as it has!