Ok, it happened again, but I nailed it this time. This morning I was riding to work, everything was cool, lights were on, then midway through the ride I noticed my headlight was out, because the blue light on the cluster that tells me the 'high beams' are on wasn't on anymore.
So I did everything I did last time, after googling the problem and coming across my own post - thank goodness because I'd forgotten what I did last time. Took me about three hours to fix it for good, here's what I did, hopefully it'll save someone some time:
Checked fuses under seat, no luck, couldn't be that easy.
Checked connections on my alarm, everything looked good.
Swapped headlight relay and flasher relay with the ones I bought last time this happened, no luck. Put the original relays back.
Had a spare left handlebar cluster for a 929, connected that one up and the same thing happened. Tossed that one back in a box and reconnected the original.
Opened up the starter switch, took it apart, everything looked good. Sprayed it with WD-40 and left it open.
Put a charger on the battery (battery still on the bike) tried starting and revving like I did in the post above, no luck.
Pulled out a cheapo circuit tester, one of the ones from discount auto for about $2 that looks like a little clear pen with a bulb in it. Found that my left flasher and instrument cluster would BOTH blink when I had the left blinker on, the key turned in the bike, and the light tester on the blue wire on the bike side of the white connector under the right airduct cover. This made me think ground ground ground.
From here I used my light tester to test the ground on the green ground wires going into my relays, checked them one by one. Neither one worked - woohoo! Found it. Took a two-wire speaker cord and did a quick test grounding both relays together and then on the bolt on the frame securing the front fairing stay bracket (just under the neck of the frame), and yeehaw, lights lights lights! Everything worked. From there I looked at my wiring diagram and back-tracked where those grounds joined, then went back to the bike and did more light testing. Turned out that the green ground wire on the bike side of the big gray connector on the wiring harness (again under the right airduct cover) was grounding fine, but the ground on the opposite side of that same connector wasn't working. That meant that the connection was jacked up.
After a lazy try at opening the connector and giving up, I sprayed WD-40 into the switch side of the connector with the bike running. A few seconds later, lights! Yeah, all done! So I closed up EVERYTHING, was closing the garage and tested one more time. Dumb headlight went out again.
Luckily I had a spare 929 wiring harness laying around, so I used that one to figure out how to get the gray connector loose. It' not hard, just have to push the clip down all the way with a little-flat-head screwdriver and pull and wiggle it loose. It was easy on the spare harness - on the one on the bike it wouldn't come loose. I ended up prying it open using two screwdrivers, one little one to press the clip and a normal-sized one to pry loose the connector. It snapped with a pop and I thought something broke, got me worried. Turned out that the ground connection had apparently melted the plastic around the metal, and remolded the plastic a little bit on the inside. Luckily there was no big damage inside. I used the little screwdriver to scrape the connector for the ground wire clean on both sides, cracked loose the little bit of remolded plastic inside, sprayed it down with WD-40, and cliked it back together. Once again, lights!
The moral of this story is if this happens to you, try the "Try"'s listed above, should be one of those. If it's not, would be cool if you added the answer to this post