I created a VSCode extension recently (awalsh128/vscode-keep-sorted) and learned a lot of lessons trying to set up the test environment. What I wanted was:
Tests that run against a real VSCode instance, not mocks. An immutable test workspace, so every test starts from the same files. A log file per test, so I can see what the extension did when a test fails. Principles # Some things I learned the hard way about testing extensions.
Pitfalls in Production # Being able to monitor production services for potential errors is critical. Especially if the services are dependant on external resources which may become unavailable or exhibit unexpected behavior. Even if you follow good software development discipline, this is always a source of concern. Think network outages, unannounced third party API changes, hosted services becoming unavailable, etc.
Introduction & Scope # When developing a new web service, you may need to do an initial spot test walk through the operations or try to break the service based on the permissible data type values. You could hand craft all the operations, with their respective arguments, into a handful of URLs but that would seem pretty inefficient. I found myself in a similar situation and wanted a programmatic way to walk through the service operations with a specific input data set. To be clear, I am not attempting to recreate a full web service testing application but rather a simple script to spot check responses and catch any bugs in a web service endpoint's handler.
