This isn't anything new! The ability to put an operating system onto a memory stick has been around for several years - it just depends on how much space is available on the memory stick and how much you cut down your favourite flavour of Linux. The memory stick needs to be bootable and the PC has to be set up to boot from a USB device.
Personally I'm not that paranoid, though I do clear my browsing history daily but that's just part of my general housekeeping when I've finished using the computer (using a program called CCleaner).
Phil West
If at first you don't succeed, use a BIGGER hammer!!
The basic CCleaner is free and is said to be better than Windows Disc Cleanup. Another free one is File Shredder, which does just that and is useful when you realize that certain files can be recovered even after they have been permanently deleted from the Recycle Bin. If you Google the names you can judge for yourself.