From the original SynthMaker forum (2004–2014), with its members’ names and dates.
trogluddite ·
Hi All,
Another instance of the Green triggering system being a bit too lazy has been uncovered. It should be pretty benign in most cases, but it could potentially cause a problem if it is essential for a stream section to react to every single green/MIDI event, and the changes are coming thick and fast.
Here's a schematic that shows the problem...
...the code block is looking for times when the input jumps up by more than one - indicating that a step in the upwards count has been 'skipped'.
Note that it is not the counter or the green parts which are skipping a beat - just that the stream section is not being updated with the new value quickly enough, and so doesn't see every stage in the count. The proportion of skipped inputs increases as the CPU load or the rate of green events go up.
More details on the original thread HERE.
I have contacted OutSim about this for explanation/advice/fix.
Tzarls ·
Behaviour confirmed even with SM 1.1.7.
trogluddite ·
Thanks Tzarls - that confirms what I suspected; it is obviously a generic problem with the way that the green systems work.
I'm gonna also post a link over on the FlowStone forum - I suppose the problem will be exactly the same for those guys.
P.S.) He he, just noticed, the FlowStone forum doesn't have dedicated 'Bugs' or 'Suggestions' forums - Hmmm.
infuzion ·
My basic reasoning for wanting to have sample-accurate Float data input into Code is to make a dB level meter that had an average or max for only the info between screen redraws. So, every time the Redraw gets a DoIt trigger, the meter level should be only be accurate for the time period since the last Redraw trigger.
Workaround: Use the number of samples in Code that will over-shoot the ratio of samples/FPS (Frames/redraws Per Second). Thus, if I'm triggering Redraw at 16FPS, then I'd use a sample size of 3675, since SampleRate/16FPS = 2756.25. Somewhat often the same value for the meter level will be redrawn twice, & rarely the value will not be seen (due to window's times being off, but should not be an issue with my Heartbeat at 16FPS), but I'm OK with the trade off. Not excited, but OK.
Suggestion: Float To Stream primitive.
Sure, SM uses Windows timers to grab a Float value into Stream. That is fine 95% of the time. But for that 5% when you need an accurate Float value grab (eg for my meters, sequencers triggered by MIDI, buttons, etc), a primitive that will refresh the input Float value & put it into Stream started by an inbound Trigger input rather than relying on windows timers would help. Most of the time the Float input would have a 2nd connection to the Trigger in, but there may be times such as syncing with Redraw that a separate Trigger would help.
Tzarls ·
AFAIK, SM only uses Windows´ timers for primitives with time-related behaviour, like timers (duh! 🙂 ).
Anyway, could this be a good explanation of what´s going on here (the missed triggers situation)?
-----------------------------------------------------------------------------------------------------------------------------------------
*******Sample time n:
float value = 10. Float value gets read by the code box (stream input). Code box stores value: 10.
*******sample time n + 1:
Float value gets read by the code box. Code box stores value: 10. Tick sent to float. Float value changes to 11.
*******sample time n + 2:
Since windows´timer are inaccurate, the next tick doesn´t get fired after the code block is executed, but before. So, new tick is received by float. Float value changes to 12. Code read value: 12. Code compares current value(12) with previous value(10). Code reports missed trigger.
That could also explain why fast changes aren´t correctly tracked by the code block. Some value might be lost but I guess the last value is always read correctly?
What happens when you try to reproduce this bug using a higher samplerate, say 96 Khz? My guess is that the ratio of missed triggers goes down.
infuzion ·
Tzarls wrote: What happens when you try to reproduce this bug using a higher samplerate, say 96 Khz? My guess is that the ratio of missed triggers goes down.
I would not bet that without through testing. Higher SR = more CPU% = more crazy windows timers/Tickers.
Tzarls ·
Someone with an interface with higher sr please test this, mine only goes to 48Khz.
philter5 ·
i will try it with 96 kHz now.... i´ll be back with the result in a few minutes...
philter5 ·
ok guys here some first results of a simple test run i made with 96 khz:
i made 3 runs with the same settings ( Soundcard ESI esp1010, Asio Driver, SR 96000 Hz)
first i thought about adding a small module that shows a little Graph with the missed triggers but that
would perhaps fake the results... so i noticed manually (yes, with paper and pen ! 😀 ):
In every of the 3 runs i had very similar values.
After 1000 total ticks i usually had a value of 41-43 missed triggers.
After 2000 ticks the same, means round about 82-86 missed triggers
that was keeping very constant, the % of missed triggers was floasting around 4 % ( +/- 0.2%)
After 10000 total ticks a value of 420 missed triggers (+/- 3 triggers)
After 100000 total ticks: 8-12 % missed triggers
As in the first tests, the value increases the longer i let it run, but with the 96000 setting
it increases A LOT slower as with 44100. OK, this does´nt say anything, i mean this is just a pure test without any
audio processing going on but good to compare the values a little bit. TBH i haven´t really cheched the other khZ settings as well as the 96 khz but i will make a comparation chart the next days...
Tzarls ·
Interesting that the ratio of missed triggers seems to be constant up to 10,000 ticks, but when reaching the 100,000 it doubles.... But if I understand correctly (and as I guessed initially) at 96Khz we get less missed triggers than at 44.1, right? So if I´m getting this right, the idea Infuzion is proposing will not work. Why? Just think of this: the stream data is updated at sample rate. Stream primitives can´t react to triggers because their nature force them to live at samplerate. Triggers (or any other green data) can happen anytime. Triggers can happen in between samples. And there´s no way to tell a stream primitive that there has been a green value update until the next sample cycle. Even if we could update a stream primitive between samples, that value won´t be used until the next sample cycle. What´s even worse - between two consecutive samples we could have 2 or more triggers (or green data updates) and the stream "world" would never know about it. It will only know about the last update, because that´s the value when the next sample cycle happens. The only way that I can think of is having some kind of trigger (or green data) buffers, so if there is a number of values passed to a stream primitive in between samples, the next cycle the stream primitive can have a look at the buffer and process it´s data with every value in the buffer. But that could also be useless, becuase the stream primitive would only pass the last processed value to the next primitive in the chain.
I think this isn´t SM´s failure - it´s more of the way things work in the discreet realm..... there could be some workarounds, but I don´t think there´s a general "one size fits all" solution for this.
Edit - Maybe there´s a "one size" solution. Having a buffer of triggers (or green data updates) and then passing them to the stream primitives one for each sample cycle. That way all triggers (or green values) would be processed and none would be missed. Yes, there would be latency issues....
MyCo ·
I think the missed trigger problem comes from the green->stream conversion. As you might know, the green stuff is plain x86 und the stream is SSE... now to get the green stuff into stream it has to get converted (single value -> 4 item array). When the stream system would convert this on every sample process, you wouldn't be able to use S|M because the CPU usage would double or tripple. So in my theory: they convert green inputs once every bufferswitch.
The time between bufferswitchs depends on how big your in/out buffers (->latency) are. You can test this by changing the latency of your Asio driver.
I know that you think this is a bug, but it is not! The green system wasn't made for time accuracy and so the green->stream conversion hasn't to be accurate as well.
philter5 ·
okay myco if you say this i believe you, because i really DON´T have the knowledge to point something out for sure... 🙁 🙁
if i get you right right there is no need to test this longer (i allready thought this would come at some point.... 😕 )
because it´s just not the right way to go...
infuzion ·
MyCo wrote: So in my theory: they convert green inputs once every bufferswitch.
That's a good theory! I was assuming that it was once every trigger.
What do you think of my suggestion above please?
trogluddite ·
MyCo wrote: I know that you think this is a bug, but it is not! The green system wasn't made for time accuracy
Indeed - as I said originally, it is an issue to be aware of in certain rare cirumstances.
For my money, the problem in SM is not that the trigger system is bad per se, rather that certain musically important events (i.e. MIDI) are utterly reliant upon it. Gve me stream MIDI reception and transmission, and I'd be perfectly happy with the trigger system as it is.
Tzarls wrote: between two consecutive samples we could have 2 or more triggers
The test schematic is driven by a tick 100 - there should be a minimum of 441 samples between ticks - more if the CPU load is high - I just don't see how your scenario is possible in this case.
The problem bears all the hallmarks of a signal sampled at a different rate to the speed it was generated at - either an independent 'tick' for value conversion, or MyCo sample buffer suggestion seem much more likely.
P.S.) Malc has replied that he is busy with other stuff right now, but he will be looking into it as soon as he has time.
MyCo ·
@infuzion: the "Float To Stream primitive" ?
I think that it isn't possible without any major problems. The stream processing runs in the audio buffering thread or even in the VST-Host Thread. The green stuff runs in its own thread. So everytime when there is a green>stream conversion, these threads have to be synchronized.
BTW: the test-schematic uses DirectSound... I don't know how this is on the other testers computers but on my system DS has a very high buffer size (when I remember correctly it's 4kS per default), so this is propably the worst case you'll ever get. Typically the driver buffer size is below 512S and many hosts split those buffers for VST in segments for faster feedback response
Tzarls ·
@ trog: What happens if a tick gets so delayed (because of its inherent inaccurate nature) that it gets thru just before (and very close to) the next one? That way you could end up with 2 triggers (or more?) between samples. 441 samples between ticks would be right if timers were accurate, but they´re not.
Also, I don´t think greens are converted at buffer switch. That would render the system very unresponsive IMHO. I think SM works very much like Max - PD work. When an external event creates a trigger, the corresponding value is stored in the primitive (say, a green int) and the trigger is processed - the way to process it is to call the "trigger" function of the next green primitive in the chain. That next primitive gets the corresponding value and processes the trigger, calling the next green in the chain. If the next primitive in the chain isn´t green, then the trigger stops there.
When a stream primitive is executed, it just reads the value at its inputs (green or stream, doesn´t really matter) and processes them. Kind of a "setter - getter" scheme - green is setter, stream is getter.
I have an idea - why not feed the test schematic with ticks generated by external MIDI events. Those events should be relatively stable in timing. Also, those events would be presented to the schematic in a sample accurate way (because of the way hosts sends MIDI events to plugins). I´d do it myself, but my studio PC is unuseable right now.
MyCo ·
Tzarls wrote: When a stream primitive is executed, it just reads the value at its inputs (green or stream, doesn´t really matter) and processes them. Kind of a "setter - getter" scheme - green is setter, stream is getter.
And when in your oppinion a stream primitive is executed? Think about that again 😈
Tzarls wrote: I have an idea - why not feed the test schematic with ticks generated by external MIDI events. Those events should be relatively stable in timing. Also, those events would be presented to the schematic in a sample accurate way.
No they are not sample accurate. The accuracy of midi events in VST is determined by the VST plugin itself. If it needs it, it caches the events and process them per sample offset. If the plugin doesn't want to be so fussy, it process all midievents right at the begining of the bufferswitch. I think that's what S|M does, because there were Note-On/Off problems in previous versions that showed a behaviour that this implementation typically has (very short notes stuck when NoteOn and NoteOff appear in the same bufferswitch event).
Tzarls ·
MyCo wrote:
And when in your oppinion a stream primitive is executed? Think about that again 😈
The stream primitives are executed once per sample - I don´t get what you mean. SM receives a buffer of samples. It then does a loop of (number of samples in buffer) iterations. On each iteration, a single sample enters the schematic, and every stream primitive is processed in the "right" order.
MyCo wrote: No they are not sample accurate. The accuracy of midi events in VST is determined by the VST plugin itself. If it needs it, it caches the events and process them per sample offset. If the plugin doesn't want to be so fussy, it process all midievents right at the begining of the bufferswitch. I think that's what S|M does, because there were Note-On/Off problems in previous versions that showed a behaviour that this implementation typically has (very short notes stuck when NoteOn and NoteOff appear in the same bufferswitch event).
What I mean is that MIDI events, as passed by the host, are all related to sample positions (the delta value associated with every MIDI event). So SM knows exactly at what sample each event should be processed. Even it if processed all of them at sample 0 of the current buffer block, they would still be aligned to sample positions. In other words, no inter-sample events.
Wrote something here? To have a post removed or your name hidden, write to [email protected].
SynthMaker