I did eventually get around to taking one of my DIMMs out and the computer was running perfectly with just the 2GB, even if it seemed ridiculously slow after using it with 4GB. It had been running for days with no crashing so I swapped the two over last night.
Everything was fine, watched some stuff from it on the HTPC, played Trackmania for a bit, got worried that nothing was crashing. Just tried to open Computer Management and it BSODed straight away so I'm sure it's a faulty DIMM.