Possible, but that's not an overly thick barrel nut to begin with. Start filing flats, and you may end up with a barrel nut that crumples, twists, or rounds off when you apply force. I've done similar before, on muzzle device, and had it work for me, after I removed enough material, and cut enough grooves in it, to reduce the force necessary to break it loose.
From the previous damage to the barrel nut, it's obvious that it was on there overly tight, or with threadlocker, or both.
Guy brought an Anderson AM-10 (Anderson’s AR10) receiver set and all Anderson parts for me to assemble.
I’ve never had that much of an issue doing a build.
Some of the parts in the LPK were for an AR10, but some were not, so I had to substitute some parts.
Here’s the disconnector that came with the FCG:
I replaced the disconnector, installed the FCG, and it would not function test. You had to apply about 18-20lbs of pressure, and pull the trigger 3-5 times, to get it to fire. I replaced the entire FCG, and I’ll diagnose the crap one later.
The head on the pivot pin wasn’t machined correctly and I had to modify it to fit.
The holes for both the bolt catch spring and buffer retainer spring were packed with blasting media. Soak and air compressor wouldn’t free them, so I had to use dental pics and drill bits that I turned with my fingers to clean them out.
The threaded hole for the pistol grip screw was not threaded properly. I have the tap for that, and corrected the issue.
The carbine buffer spring would not compress enough to allow the bolt to lock back. No, it was not a rifle spring. It was a correct-length carbine spring, and the buffer was correct as well. I replaced it with a Sprinco “orange” (carbine length intended for a .308) and it function checked fine.
All in all, the build took me 4-5x longer than a quality build would have taken me.