I've set up logging on my desktop PC (Windows 7 Pro), then closed the GUI, then deleted the task via Task Scheduler, but on reopening the GUI everything works as expected. Clicking Display Task Info on the menu gives a message saying the task does not exist, as expected.
I simply can't recreate the issue, also the "File not found" error is a little odd, it should be using SCHTasks to check if the task exists, so should not have any file not found errors, it would be useful to see a screen shot of this error to pursue things further.