I am always looking for good tips on world building, I like this approach. As I myself have several hundred hours ahead of myself likely before my world will be done its always good to see another’s view on how they do it !
Instead of starting small with a town or village, start even smaller. Think of an artifact. Let your imagination wander. The artifact could be a magic item or just an item of great importance.