The documentation was misleading me. You can indeed host a wms server given a sqlite cache.
My problem was in my config. I specified the path wrong possibly in globals (used windows style \ instead of /). I got it working when i saw the folder structure it generated at the root of my disk.
I ended up with this config (my drive letter is E and using win 10):
- name: my_cache
title: my map
meta_size: [1, 1]
At the location when i relied at global base dir location the cache was created at the root of E.
The my_cache folder structure has a folder called sim_webmercator in which are the tiles and tile_locks folder. Mapproxy generates both if there arent any.