The ability to problem solve and then brute-force your way through the effort to implement a solution is pure magic.
I'd love to hear more stories like this from everyone here. Regale us with tales of your legendary battles!
quotingThe electronic project that I am most proud of from my engineering days, was also the most useless thing I ever built. I was reminded of it amid all the censorship in Brazil, because it’s about Embraer avionics, which are Brazilian.
note1n9v…tafc
A short story.
The background context was that I never considered myself to be a great engineer. I was more interested in finance but had gone into engineering instead, and basically what makes someone great in a field is whether they have the passion to do it in their spare time or not. The best engineers, after school or after work, go home and do more hobbyist engineering stuff. Like, they just live in it. I spent my spare time on financial research instead. I liked my engineering work, but was only a design engineer when I had to be, basically. My passion was elsewhere. So I knew I'd likely eventually go into engineering finance/management in that kind of hybrid role.
I went to work in an aviation simulation facility after college. Rather than being used for training new pilots as most airplane simulators are, this facility was used by corporate, academic, and government researchers to test new technologies and procedures with experienced pilots. So the simulators were more customizable than training simulators, but were less polished. They were always these cool work-in-progress machines.
We had Airbus, Boeing, and Cessna models in the early days. And we had a half-finished Embraer, simply because the manager of the simulation facility just wanted an Embraer due how cool and different they are. His higher-ups in the broader parent organization told him there wasn’t enough market demand for testing in Embraers and wouldn’t give him a budget for it, but he kept finding ways to gradually accumulate the parts anyway. Eventually he had the simulator shell, and genuine avionics, but still had to put it all together. The Embraer avionics and flight controls were just so much more interesting and sleek than the other types.
In our Boeing and Airbus simulators, we had genuine avionics, but interfaced them into industrial automation control logic and PCs, rather than what they normally connect to in the field. And it wasn’t hard because they used some standard communication protocols like RS-422 or RS-485. All the buttons and dials were easy to connect to automation control systems, and the serial communications were things you could buy a PC card for.
But the Embraer sat unfinished for years partially because there was no budget for it, and partially because it didn’t use those standard serial protocols. The serial communications coming from their flight panel and throttle were some custom method that none of our equipment could read. You could just rip it open and re-do the insides to talk to a PC more easily, but the manager forbade that; he wanted the Embraer avionics to remain pristine. We had incomplete documentation, and virtually no budget or labor to put into it beyond the pieces we already had. It was this hacky project. We didn’t even really have electronic design software other than a lapsed PSpice license, since we rarely did any sort of custom circuits and instead would interface various devices into automation systems.
So, getting the Embraer up and running was always on the manager’s pet project to-do list but not making much progress. He would occasionally put an entry level or intermediate level engineer on it for a few months and they’d eventually give up, leave the organization, or go back to something more economically relevant. It went on like that for years; it became the Kobayashi Maru for electronic engineers in this facility.
After my first couple years working there on the Boeing, Airbus, and Cessna, the manager was eventually like, “alright, I want you to see if you can get that Embraer flight panel connected.” Little budget, just me in the lab with an oscillator and multimeter. I knew this would happen one day- the Embraer avionics were always kind of looming in the lab, next to half-finished failed creations of my predecessors that tried to interface with it, knowing that eventually I’d be put to work on this thing. And I still had other duties to do, so this was like a budget-less side project I had to spend part of each week on, with insufficient equipment and documentation and with the knowledge that the Embraer simulator might not even be used if I somehow succeeded.
So I dove into it. The Embraer flight panel had a series of buttons, but unlike our other flight panels, they all went through a parallel-in serial-out (PISO) shift register. So they came out of the device in a fast custom serial communication stream. An external device had to give the flight panel a clock signal, and give another CLR signal that would be low for 24 milliseconds and high for 1 millisecond, and during that 1 millisecond there would be 256 clock signals, and the flight panel would output all of its current states of button and knobs within those 256 clock signals within the millisecond, every 25 milliseconds. And the CLR signal had to be shifted some microseconds away from the clock signal so that the signal edges of their square waves don’t occur simultaneously, which would cause reliability issues. The incomplete documentation we had only roughly outlined this; I had to iterate a bunch of oscillator tests to see what successfully worked as an input, and then more fully document where every button and dial showed up on the output data stream once I got it flowing.
And then once I figured out how it worked, I had to figure out how to connect it to a PC and/or our standard industrial automation interface.
At first I tried one of our 2 year old microcontrollers that we had on hand, which as of this writing are now nearly 15 years old so not the current generation stuff. I looked to see if I could program it to send in these signals and then read the signals coming from it, but it wasn’t nearly fast and precise enough. This project needed lower-level design than something software-based.
I wasn’t sure if FPGAs of the day would be fast enough, I hadn't worked on them before, and and we didn’t have any anyway. So I went the raw logic route instead which would be annoying but cheap and guaranteed to work. I researched crystal oscillators, logic gates, counters, flip-flops, and serial-in parallel-out (SIPO) shift registers. I then used them to build this whole digital clockwork circuit from scratch to basically reverse everything that the flight panel was doing. It would use a crystal oscillator as the initial frequency, and from that would generate the clock signal and the CLR signal to send to the flight panel, and would run the timing for a bunch of SIPO shift registers so that this stream of serial data coming out of the flight panel would load into the shift registers, then get frozen in place for 24 milliseconds which was fast enough for our slower industrial automation interface to read the data which was now in parallel form.
Then I bought all the cheap logic pieces, soldered together test circuits, troubleshooted it, and got the thing working. I came in alone on the weekends to do it since I had little time in the week; I was just caught in the flow of making this work. Rather than viewing it as annoying as I did early on, I now viewed it as something I had to beat.
And then the Embraer throttles were the second difficult piece. Rather than a normal position output as our other simulator throttles did, the Embraer throttles basically outputted sign waves, and as you moved the throttle, which was really well-crafted and smooth, the two sign waves would overlap to varying degrees. So I had to map that all out, build a circuit to read these sign waves and send that position data to the slower industrial automation interface in a way that it could read it. I then soldered together the circuits, and got that working too.
I then wired up all the more standard electronics in the Embraer which were not that different than the other simulators. And so, I was finally able to go the manager and say, “the Embraer’s flight controls and avionics are all connected now! ^_^”.
I remember being so proud, like, “oh shit, maybe I could actually be decent at this.” And the custom circuit interfaces indeed worked for the next decade and counting without maintenance, perfectly reliable. We then got the rest of the simulator fully set up.
But… the Embraer simulator was only used like one time, barely. No research teams wanted to test new technologies and procedures on Embraers; they would always test things on the Airbus and Boeing simulators. So this awesome simulator, with so much work poured into it, was a complete waste of time, which is what the managers’ higher-ups had told him from the beginning.
I went on to do a bunch of other projects including a business jet simulator and a helicopter simulator and an upgraded redesign of our Cessna before I went into management and out of design engineering, but nothing was quite as hacky as that Embraer simulator, so it was always the one I would look at fondly and be like, “I can’t believe this useless thing is still running.”
Rather than feeling sad that the work was wasted, I instead always kind of had a “journey before destination” mindset to it, like it was the most frustrating project that I had brute-force succeeded at, from research to design to physically putting it all together to troubleshooting, and gave me a bigger sense of achievement and got me into a higher state of flow than other engineering projects I had done. To this day, I always think of Embraer planes warmly.