Writing a motion cueing software from scratch.

Notes upfront:
You can download the latest version of the software here.
So far, it supports X-Plane and DCS, but could in gereral support any simulator.
Output to the hardware is still limited to linear actuators. Crank arm support could be added with minimal effort.

Feel free to ask anything down in the thread.



XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Original Thread starts here:
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX



Last November my actuator was nearing completion and I started to look into Simtools/BFF/X-Sim to see how I would get them to interact with the AMC1280. Up to that point I never paid much attention to the software side of things, because I was simply assuming that these programs would do what I expected a motion cueing software to do. I was surprised to see that apparently „motion cueing“ can be understood in a wide variety of ways

Now, I don’t wanna step on anyones toes here, but as the title suggests you can pretty much guess already that eventually I decided that neither Simtools, nor BFF, nor X-Sim allowed me to do what I wanted with my 6DOF Stewart platform: Actual full-on 6DOF acceleration onset cueing.

So before I go into further detail, please note that I don’t want to bash any of those programs, or their communities, or contributors, or developers, or users in any way! This is really important to me! I have gotten SO MUCH from these communities. So much help, support, advice, words of warning, entertainment and inspiration. Most of all, from all you guys here!

However, I would like to explain why I chose to not go the route that so many of you follow with great enthusiasm, but instead write EVERYTHING from scratch.

As far as I can see, there are three options available to get a 6DOF platform moving:

• Simtools
• BFF
• X-Sim

Simtools (correct me if I’m wrong!) does not provide actual acceleration onset cueing. You can read data out of sims, and then use that data to move certain axis’ , but It didn’t allow me to coordinate those signals in a way that leads to what I think the magic of acceleration onset cueing at its core is about. I mean, no frequency splitters, differentiators, integrators, arithmetic modules, conditional processing, limiters, threshold functions, predictive cueing, custom transfer functions, tools to analyse the frequency response or impulse response of the system…

X-Sim has some of those elements, but afaik is no longer being actively developed(?) and still limits me to be a user much more than a developer.

BFF looked like the most promising option to me for a long time. It certainly has many of those elements implemented, but very limited options to configure them to my needs. There are ways to graphically plot the signals in the time domain for example, but only rudimentary. It is the only software however that actually has a motion cueing algorithm implemented, albeit with limited means to adapt it to my use case.

All three are commercial products that try to cater to a wide rage of customers (understandably) but they give me limited freedom to adapt them or my own project. And most importantly: all three are closed source!

Please don’t get me wrong: I don’t think I could come up with anything that would ever replace these programs. I just want to come up with something more suitable to my needs.


So, I decided to write my own from scratch


Please be aware, that this is NOT A TUTORIAL!!! I’m not saying: „I know how it’s done, come follow me!“ By no means! I have absolutely no clue what I’m doing here, but I’m sure I will learn along the way. I have very limited programming skills, about 16 Months ago was the first time I ever heard the word “Arduino“, 4 months ago was the first time I ever wrote something with a clickable button, and the purpose of a callback function is still a bit vague to me. I have never seen a university from the inside and was a less then mediocre student in a public school in rural Germany.

That being said, I think I have a couple of good ideas on how to implement a motion cueing software the way I think it should work (for me!) and I’m publishing this for others to learn from my mistakes, draw inspiration, give advice or simply be entertained by my futile attempts to delay my impending failure just a little bit every day


So, what even is a motion cueing software? Well,… actually the software is only one link in the „motion simulation chain“. To get the big picture:





Basically three steps here:

1. Extraction: Get the data out of your sim into the motion-cueing-software.
2. Processing: Crank it through some black voodoo math-magic
3. Injection: Whatever comes out on the other side will need to be injected into a motion system to drive the actuators

I will go into more detail on the extraction and injection at a later time, because I think the processing is probably the most interesting part.

I do the processing in two major modules:

The first is the motion cueing algorithm which takes the vehicle state as exported by the simulator and turns it into a platform state.

The second module then uses inverse kinematics to turn the platform state into the 6 actuator states.





Of these two modules, the motion cueing algorithm (MCA) appeared to be the most challenging to implement. I mean, initially I didn’t have the faintest idea of how to implement anything really in code, but in all other areas I was somewhat confident that I could understand the problem, follow tutorials, find documents or come up with something myself. It wasn’t until I fully understood how to implement the MCA, that I started the actual coding.

Btw, there is no such thing as THE motion cueing algorithm. The „One to rule them all“. There are a variety of ways to create the illusion of motion to an occupant and essentially each is the solution to a single, very well defined use case. And it is always a compromise between different, sometimes contradicting, requirements. However, you can define certain properties you want your MCA to have and in one form or another all MCA designers will have made considerations to what extent they want these properties implemented.

A (non exhaustive) list of what I wanted in my MCA:

• Driver-in-the-loop system
• Real time (no pre-processing)
• No look-ahead functionality
• No predictive cueing (…yet! But possible later)
• No adaptive cueing (…yet! But possible later)
• Filter based approach (washout through HighPass-LowPass combination)
• Tilt coordination (Pitch-Surge and Roll-Sway)

This is what I could come up with just now off the top of my head. I’m sure there are more and I will add them as they pop up. Btw, it’s not like I invented those. I gathered them from (unfortunately very few) research papers I found online. The Wikipedia page on Motion Simulation provides some great resources as well. Especially the section on false cues is worth reading and understanding.

I am already a few months into the development of this, but I would like to publish things in chronological order. So, I will report here step by step the path I took and what I learned along the way.

Dirty

Fourier:

I’m not gonna bug you with in-depth tutorials on Fourier or LaPlace transforms (mostly because I wouldn’t know all too much to tell myself), but these two videos are a „must watch“ to get an intuitive understanding of what is meant when people say „high frequency content“ or „low frequency content“ or „every signal can be represented by a Fourier series“.
SmarterEveryday
3Blue1Brown


Digital Signal Processing

There are a hundred different places to start, but I figured that a quick brush-up on those digital signal processing skills we all learned in middleschool might be needed if you want to understand what is going on inside a Motion Cueing Algorithm (MCA). No need to listen to 40hrs of YouTube lectures about LTI systems (like I did) and I will not quiz you about the difference between a Fourier- and a LaPlace-transform.

What you should understand however is that in digital signal processing (DSP) we are dealing with discrete-time-signals (as opposed to the continuous-time-signals in those videos). So, no matter how high your frame rate, you will always have just a bunch of discrete values. Nothing in between. No gradients! …like ever!

That is going to become an issue when we need to create the derivative of a discrete-time signal.

Think of a „system“ as a black box. A signal goes in, a signal comes out. And what that box does to the signal is written on it in big letters. You don’t need to understand HOW it’s done, but you should have a general understanding of WHAT is being done.

For more info: LTI systems

Low Pass Filters:

Probably the simplest low pass filter would be a simple moving average filter (SMA). In a nutshell: It outputs the average of a certain number of previous inputs. The higher that number, the more that filter will act as a low pass filter. The more high frequency content is being filtered out.



A weighted moving average filter (WMA) is the same thing, just that newer values are being „weighted“ more than older ones.




An exponential moving average filter (EMA) is kinda similar to a WMA, but the values are depreciated by a defined factor after each new value arrives. Old values never ever go out of scope, but are just depreciated into oblivion. Sounds more complicated, but isn’t. This one has less delay between the input and output and is also easiest to implement because it does not even need to keep track of more than a single history value.



My filter of choice is mostly a double exponential moving average filter (DEMA) because it has even less delay at similar smoothing properties than the EMA. Also relatively easy to implement. It does overshoot slightly though (It’s underdamped).


High Pass Filters:

High pass filters gave me headaches for weeks and I have settled for a pragmatic solution: If you take the original signal and then subtract from it the low pass filtered signal you should be left with the high frequency content.

I know,… I know,… the low frequency content is delayed, yes! But a DEMA is not THAT much delayed to render this method useless. If someone can come up with a better method, I’d be happy to learn. Seriously! Or even better: Can anyone provide me with a code snippet in C++ or C# of the simplest-possible high pass filter? I’d appreciate!

The advantage of this method of using the "compliment" to the low frequency content as high frequency signal is, that I do not need to coordinate the two signals when they act on the platform in different channels. It will become more evident in the future, but for now, just note that for proper tilt coordination (longitudinal and lateral) the illusion of motion is being started with the high frq signal in the translation channel and then kept alive with the low frq signal in the rotation channel. Key is to apply one when the other fades away. Using this complimentary method will basically ensure you're not too far off in that coordination effort.

Are differentiators high pass filters? NO! But they have properties that make them act similar. Think of them as filters that let the entire frequency spectrum pass, but lower frequencies are attenuated more than higher frequencies. The attenuation is inverse proportional to the frequency, so there will be way more high frequencies left in the signal. And I think it introduces a 90° phase shift.... but that is speculation.

Dirty

Great stuff @Dirty most of it went straight over my head lol

Isn't this close to what you are trying to achieve https://www.xsimulator.net/communit...face-for-linear-and-rotating-actuators.12859/ i know it still gets the data from simtools but its got all the filters and stuff on it?

p.s. After writing the above thinking about it I'm sure you've already seen it lol

So if I get right, you want to include the computer software in the motor control loop so you can manipulate the motor accelerations?

Wouldn't be easier if you just "export" these acceleration calculations along the position data so the controller would know where to move the motor and with what speed? Assuming that is what you are trying to accomplish....


reading a little bit more.... You would have to include the simulation game itself into the control loop... as the current simulation games just export the accelerations or position one-way only with no external feedback....

Yes I saw it! But, boy, did this thing develop! I haven't checked in a while, I have to admit. It appears to have matured quite far in the past months!

And you're right, it looks very similar, and gets it's data from Simtools. That is not a bad thing, quite to the contrary: It means @pmvcda will have a huge base of supported sims right from the start through SimTools

As far as I understood, his Hexapod interface, as well as this interface are focussed on that second step of the process: Inverse kinematics, or "turning the platform position into actuator positions".



What I want to focus on is that first step: Calculating where that platform should be in the first place! I think THAT is an area where DIY based motion simulation has HUGE unfulfilled potential.

Oh no That is not what I meant

1. The Sim exports its motion Data to my software
2. I do the processing (motion cueing) and calculate where the platform is supposed to be
3. I do the next step and calculate how long each actuator will have to be (inverse kinematics)
4. I send the data (basically only six values) to my controller where I have a PID loop to drive the actuators.

That is a one way street! No feedback from the controller to the software. Only a closed loop between the controller and the actuator. I hope that clears it up

The "adaptive cueing" I mentioned is an option I find interesting, but not a must-have. And it would not require feedback from the controller.

I'm sorry but I struggle to understand what is different from what it has already been done from the existing 6DOF plugins. All have these exact steps...

Yes, they do. But step number 2 is what I am not satisfied with I want more freedom to route those signals any way I like. Through the filters that I code. I want to determine what happens and make changes any way I want. That's why I'm willing to put in this work into this.

Hi,

Making part #2, "inverse kinematics" is the most basic part of it all. Maybe for rotating actuators it gets more complicated, but honestly, it's very basic and fast.
Just use a quaternation to rotate points and measure distances. Nothing special.
Rotation should be real 3D, not just on a plane. And to make it fast, I'm using some old tricks of the 8 bit gaming time...

I have implemented the filters and thought that using the mix of simtools with the interface could produce a similar effect to a motion cueing algorithm.

But after seeing people struggling with setup, the best is to put everything inside the same basket.
So my next version of the software (this week) incorporates the motion cueing algorithm, the classic one:



In a simplified view:



This is my first step. You can make a search for motion cueing and you will get many papers with adaptive algorithms and much more...
I decided to use the filters, as I have, and made an interface like this:



Relative to current version, washout was modified and we can now mix sway and surge in roll and pitch.
Might have to much filter options, but we can play as we want...



.......

I have EMA DEMA TEMA filters and combine them for washout.
I use mainly EMA, don't think DEMA is necessary.
I use low pass as integrator.
Now I have Washout=LowPass(High Pass(value))
High pass is: High pass= value - Low Pass(value)

In the image, we specify the number of samples used in the EMA filters,
At left, for low pass or lower cut off on band filters
At right the number of samples for high pass or upper cut off on band filters.
Those two number of samples are also used in the washout high and low pass filters

It's pretty fast calculation.
Slow is to show and store the graphics.

In my rig I'm sending speed to the controllers (ESP32), because I have two controllers and I have no communication between them.
I'm calculating speed on the interface by receiving current position from the rig and compare to pretended position to obtain desired pose.
Making this, I also have PID in the interface and I can change and tune everything real time.

The disadvantage is the serial communication by USB... I have to send and receive data and a long cable makes things slower. But it works.
I got a STM32F4DISC1 and thought of making everything on the board, but I'm still trying to find time for that...

The advantage is if you want a specific pose, all actuators move at a speed that makes them arrive the desired position at the same time.
In reality with the frequency of updates we have, it's really not noticeable, but if you go from one pose to the other and stop, there's a good difference in behaviour.

Yes, that's exactly what I meant:


If I had known that you were working on an actual Motion Cueing Algorithm I would've contacted you weeks ago. Chances are that this will render my whole project pointless Which I'm totally fine with

I am using the Unity engine to do all things that have to do with manipulating objects in space. Initially I was trying to do it all in a WinFormApp (C#.NET) and use Unity only for a graphical representation, until I came to the point where I had to write myself a small lightweight quaternion library, which is not exactly my core area of expertise.

Eventually I decided that I would implement the entire program in Unity, which...
a) spared me the burden of having to do the quaternion operations myself
b) gave me a fully integrated graphics solution
c) allowed me to play around with different MCAs as the Unity hierachy tree basically IS my MCA structure. I can change/modify/add/rearrange elements through drag 'n drop.
d) allows me to observe/adjust literally every value on-the-fly while the application is running. I can even change the time scale and watch everything in super slow-mo (like 0,1 FPS)
e) gives me access to the Unity Profiler to watch my program working frame by frame

Initially I was anxious to use Unity because I was afraid it might come with a costly overhead load. But so far it only occupies 75MB of memory and 3% of CPU and runs at a whopping 1400FPS for the internal cycle. Visuals of course 60FPS.

There is one thing I am 99% sure I will implement, and that is a conversion from a center of gravity referenced motion data stream into an occupant referenced motion data stream. Taking care of the fact, that the driver is not sitting in the vehicle CG and experiences slightly different forces than the sim exports. Might seem only marginally important for a Go-Kart sim,... but is vitally important for airliners. Even in racing cars you can feel the difference between touching a curb on the driver side or the other.
That would be a nice module that I think we could share if you're interested.

So, I guess we have been working on the same problem for a while. I will keep posting what I do here and I will sure keep a close eye on your project. I will eventually put everything open source.

Dirty

That's exactly what I was thinking lol obviously I'm joking everything here gone way over my head.

Glad your both on the same page should make it easier to bounce ideas and knowledge to each other

Lots of good stuff going on here lately great to see

Well, in my program, you can change the center of rotation, so in a car you can somehow simulate what you would feel if you where seated in the back seat.

The problem here is that you are not sure of what info you receive from sims.
While in aircraft they are more serious about it, in racing games, you are not sure of what you receive.
Is info related to car CG, driving position, center of traction or any other?
Imagine that in a game, instead of car info, you got the memory address of the antena vibrating... (exaggeration)
We have no idea and can't be sure of what they say, even with memory maps.

In my program, if the data is relative to the seated driver position, center of rotation should be adjusted to be on your seat position.
If I want to know what I would feel in the back seat, a good approximation was to move center of rotation to the front, around 70 cm.

So, it's all an approximation. To be perfect, we need full support from game studios and there we have info to reach near professional sims.

I don't know if it's worth to calculate forces on a new position when we don't even know what we receive. But I'm probably wrong.
The advantage would be to use the rig limits in a better way with motion cueing.

My next step is to make the program work without errors and that everyone understands how to use it.

Oh, lots of good considerations in that post. It's just a pleasure to read from you.

Indeed, it all hinges around the question of where on the model the exported data is being picked up from. In flightsims I would almost certainly bet my butt on it coming right from the CG.
In driving sims, I'd expect the dispersion to be a bit wider from "games" to "hardcore sims". But I have to admit, todays driving sims have come quite a long way from the arcade halls of the 80s. So even in that genre I would assume it to come from the CG.


Shifting the center of rotation on your platform appears to be a clever way of approximating that effect. And correct, it would chip away from your geometric envelope, so I would only use it for rather small distances. Sorry, widebody airliners ;-)

Dirty

Correct me if I am wrong but since we have the 6-dof IK plugins, this kind of "classic washout" could have always been constructed from scratch in x-sim or simtools using simple axis mapping with simple LPF and HPF.



Although it's very cool to have it AFTER the kinematic side I think, there is a big benefit being able to build those BEFORE mapping them on the degrees of freedom, which would give us in some cases more flexibility while mixing different effects.

@yobuddy For sure, it would be very useful to develop inside Sim-Tools more advanced filtering options and tweak the limits of the existing ones.

A good example is the telemetry coming from X-Plane. Besides the rotational positions, we have access to rotational accelerations which is very interesting to mix with the positions.

Applying the filters on the effects directly (SimTools or X-Sim side) and not on degrees of freedom (inside the plugin) could give the user more control of motion filtering on what goes where and with which kind and amount of filtering.

Best

That sounds interesting. I think that could approximate an actual motion cueing algorithm to a certain degree.

I second that!
It's certainly not that you couldn't get anything moving with SimTools as is. It's just that I was looking for more freedom to explore options. The down side if it would be, that it would probably require a much deeper understanding from the user to setup and tune.

Since I'm pretty sure that this wasn't gonna be what the majority of people were going to be interested in, I was happy to start on my own path

Motion components

Trying to think of „motion“ as a whole is pretty complicated, at least for me. Just too much going on at the same time to be able to do anything useful with it. But we could break the motion stream apart into its components.

Probably trivial to most, but I’d like to mention it none the less for the sake of completeness: Any motion is just a combination of translations and rotations. Any changes in motion (aka accelerations) are just a combination of changes to translation-rates and rotation-rates.

So instead of looking to find an algorithm to process all of the possible motions a vehicle can perform, we just need to find a way to process translations, rotations, accelerations and angular accelerations. That sounds a lot more accessible than „motion“ as a whole to me.

In the end we will then send commands to the platform to display all processed motion components simultaneously (in superposition) which will be interpreted by our brains as actual motion

Translations are movements along the vehicle x/y/z axis. Rotations are rotations around the vehicle x/y/z axis. (Unless I indicate otherwise, I’m talking about motion in the vehicle reference system).

These axis are perpendicular to one another, so for translation one axis does not influence any of the other axis. The order in which we execute those motions does NOT matter.

Rotations unfortunately are not quite as straight forward. Here the order DOES matter.


Euler Angles

Leonhard Euler, a Swiss dude from the city of Basel, came up with a set of angles to describe the orientation of objects in space, and as long as you respect the Euler-angle sequence of…

1. Yaw
2. Pitch
3. Roll

…you can use them to describe any orientation of an object in (Euclidian-) space. However, calculus was not invented at the time, so there are a couple of drawbacks to the Euler Angles: They do not allow for transient states. Well - ok - technically they do allow, but they give strange results when you use them to describe motion as you intuitively would:
Imagine you are in a plane on Heading 355° and you turn right by 10 degrees to a heading of 005°. What was your average heading? What was your heading rate?
Even worse: Imagine you do a loop and shortly before your nose crosses the vertical you have a heading of 000° (or is it 360°?) and as soon as you cross the vertical your heading will jump to 180°, your bank angle will flip to 180° (or is it -180°?). A tiny rotation of the vehicle around its lateral axis has generated huuuuuge changes in two seemingly completely unrelated axis.

And when the heading flips over from 000° to 180°, was that a righthand or lefthand turn?
And what about the moment your nose actually is pointing directly up (pitch 90°). What is your bank angle? I can tell you: It’s not defined!!!
That’s a nightmare if you want to drive a motion simulator with this data.

Bottom Line: Euler angles describe the steady state of an object in an instance of time, but are not useful for motion. For that, we can use…


Matrix Notation

Instead of describing orientation in space through angles, it makes more sense to describe them with three vectors. Basically 3 arrows in space, (perpendicular to each other) representing the direction of the longitudinal/vertical/lateral axis of the vehicle. I have actually written myself a little linear algebra library starting with functions as basic as the dot product and cross product of vectors and reached a point where I could manipulate objects in 3D space. That was quite uplifting Just a few months after starting pretty much empty handed!

At that point I was pretty sure that I could actually do this with the simple means I had available.

Two problems still:

1. I read literally everywhere that I should use quaternions instead :-/
2. I needed a way to graphically present those objects, as up until then I only had them in computer memory.

Quaternions

<sarcasm>
Quaternions are straight forward. I mean, what could possibly be more intuitive than creating a 4 dimensional sphere in 4D-hyperspace that represents the points of the object we want to manipulate? All we need to do then is rotate this sphere in hyperspace so that we can project the resulting points back into 3D space! Is it really THAT hard to get this into your head?!? WHAT PART OF „LETS EXTEND THE COMPLEX NUMBER PLANE INTO 3 DIMENSIONS TO CREATE A NEW NON-COMMUTATIVE NUMBER SYSTEM THAT REPRESENTS ROTATIONS“ DID YOU NOT UNDERSTAND?!?!
</sarcasm>

Seriously, I have attempted to vaguely sketch how I would implement quaternions but found it cumbersome (to say the least!). I never reached a point where I actually got any usable results and that was pretty frustrating. Luckily, It occurred to me, that I am certainly not the only person on the planet faced with the problem of wanting to use a computer to manipulate objects in space. Which brought me to the…


Unity Engine

To me, this is the one-stop-shop to work with now. It solved all of my problems in one: I can generate a GUI with it, create different objects that represent the objects of my MCA. Let all of them interact in the engine, harness the diagnostics tools, tap into the forums to get support…

I know, it’s meant to create games and not motion simulation, but computationally there is really no difference between Ogres and Elfs exchanging damage points and Streams, Filters, Servers,… exchanging motion data. The CPU won’t know the difference

As soon as I get my voice back (got a pretty sore throat!) I will make a video of it. That will make it much clearer.

“Translations are movements along the vehicle x/y/z axis. Rotations are rotations around the vehicle x/y/z axis.”

Yes, and basically, You have to map the vehicle xyz positions to 6DOF plug-in rotations, and vehicle xyz accelerations to the translations.

As the Stewart platform has a very limited workspace, additionally, you may want to map the longitudinal and lateral accelerations to the Pitch and Roll and mix them with your already existing positions in order to use gravity and cheat your velstbular system so you get a feeling of sustained braking and acceleting effect.

The issue with the yaw is, that most of the games offers only rotational positions, rarely you can get the angular accelerations. In this case, the plugin wrighter has to do some math to convert the absolute yaw to local yaw to avoid the kick 359 deg. - 0 deg.

SimTools washout feature needs also to be adapted so it can properly recenter the axis. @yobuddy promised couple of months ago that this will be done in upcoming release.

If you want more technical precisions about how to deal with yaw, talk to @value1

As he wrote a lot of game telemetry plugins, he will for sure be able to provide detailed info in that matter.

And ideally you compensate the acceleration resulting from the rotation (centrifugal forces) with a compensating force of a lateral movement.

Outputting rotational speed is somewhat common in many simulations. I guess this will satisfy the mind to be tweaked into the right state.
Calculation can be done, however my physics professor once told me: If you want nice graphs, don't differentiate, always integrate! same here