Pitch to midi-2SYN, FretKey builds

RunBeerRun2 · 28 Oct 2008

Cool, osm! I'll check it out, cuz I have no idea what most of the gui does.

nix · 29 Oct 2008

Hey there 🙂, I have gone over the signal flow a little in this thread at KVR-> http://www.kvraudio.com/forum/viewtopic.php?t=230754 I think the presets need to be tweaked to get the best performance. Let me know if you have any ideas for it , sure. There is an audio demo playing Contralogic Leto up in that thread too. Wishing contents in creation, nix

RunBeerRun2 · 31 Oct 2008

Cool I'll head on over to kvr and listen to the sample. Haven't opened this osm yet or used 2syn lately, today I was patching 2 bass amp sims and it took all freakin day to get them right! And they only had 2 patches lol.

matti · 18 Feb 2009

Hmm, interesting. You could use this with other instruments too? And voice perhaps too? Gotta test it out when i have some free studio-time again 🙂

nix · 18 Feb 2009

Let me know if you think of anything to improve it matti 🙂 .

McBarGig · 18 Feb 2009

Yeah Nix I checked that out very cool stuff indeed. Once I can get my other issues sorted I planned on looking further into this... Currently I'm trying to build a delay that has a LGFO applied to the "echo" only portion, wow...Crash Crash Crash.

nix · 18 Feb 2009

If you can't get anywhere with it upload your .osm if you like. I'm hoping I can track the whole fretboard with 2SYN in time, all collaborators most welcome 🙂 . Cheers

McBarGig · 26 Mar 2009

Nix I want to look furthe into this idea. What seems to be a problem is the attack from the string...Thoughts? KM

nix · 27 Mar 2009

I'll break it out again after this synth. I got good results by delaying the read, until after the string has settled down, but latency is. I'm not exactly sure what to try next, the biquad(works best so far) filter before the detect is key. I'll try this splitting method again, the plug here isn't that great considering I had some success getting to it. Maybe it would be good to have a variable time that the pitch detector reads, more accurate=more latency, but perhaps we could find the balance. This is pretty interesting if you have a 13 pin pick up-> http://keithmcmillen.com/stringport/index.html I'm gonna pick it up and make a midi translator. Per string will be a lot easier with what we have here. Interested further with this, I'll get back into it soon. Much regards

BertAnt · 4 Jul 2009

Hi everyone, i'm a new member here 🙂

nix wrote: I'll break it out again after this synth. I got good results by delaying the read, until after the string has settled down, but latency is. I'm not exactly sure what to try next, the biquad(works best so far) filter before the detect is key. I'll try this splitting method again, the plug here isn't that great considering I had some success getting to it. Maybe it would be good to have a variable time that the pitch detector reads, more accurate=more latency, but perhaps we could find the balance...

Hi Nix, how about using AMDF method for fast and clean tracking ? I think this is the one used in WIDI realtime mono correlator mode, and i have good results too using this method. It's very accurate at determining MIDI Note Number, the latency is hardly any problem at all, because once the string wave started to become periodic, the pitch can instantly be recognized. On low E latency is ~16 ms i think. Also this method is immune to false zero-crossing, which a major problem on zero-crossing detection.

nix · 4 Jul 2009

I'll look into the method, thanks for the heads up. Do you know how to implement it at all? I'll google now. I've been playing with this since last post and it seems fractionally better using 2 biquads in series to make a more elliptic filter. I'm curious of this AMDF, thanks and welcome! Cheers 🙂

nix · 4 Jul 2009

Does anyone have any inkling how to this polyphonically with FFT? I'll up my latest soon. It's simpler. Making the hop length higher helped. Cheers

BertAnt · 4 Jul 2009

nix wrote: I'll look into the method, thanks for the heads up. Do you know how to implement it at all? I'll google now. I've been playing with this since last post and it seems fractionally better using 2 biquads in series to make a more elliptic filter. I'm curious of this AMDF, thanks and welcome! Cheers 🙂

Thanks for quick reply and the welcome ! 🙂 Well, originally it is a more like FFT, which i think is a discrete equation, it probably means sampling the sounds for say 10 ms then performs the calculation for each sample. But, since i'm kinda noobies with math and DSP, i figured out a 'simpler' way with the help from my friend, using a multi-tap delay (i use four 12 tap delay which can covers entire fretboard nicely). Then simply subtract the audio signal with the delayed version, with each tap's delay time is equals each note period (f.i. A4 = 440 hz, delay time should be 1/440 sec for detecting this note). Finally measure which tap has minimum amplitude, and there you go, that currently minimum-amplitude tap time is the period of the note.

For googling, AMDF is related closely to Autocorrelation, in fact it's a variation of this well-known technique, so any autocorrelation literature almost always has parts describing AMDF as well, the best place to start is Wikipedia i think. I prefer AMDF over conventional Autocorrelation because it's easier to 'spot' the dip rather than peaks. Here's a link :

http://cnx.org/content/m11714/latest/

Unfortunately i'm just starting out in SM, i only just at getting familiar with the function and how it works, and the way that i described above is implemented in other modular environments. Attached is the MIDI output from electric guitar connected to my AMDF pitch tracker, no hex pickup used.

Sorry if i'm making it too long 🙂

Cheers ! 🙂

nix · 4 Jul 2009

Woah! That sounds great=perfect. I'll go over your post a couple times and see if I can understand. What environment do you usually develop in? I'll bell my friend oddson and see if he grasps the technique. He is the one that wrote the code for the 0 crossing counter and maybe we can implement in SM. It sounds a lot better than what I have here. Thanks for sharing your method 🙂 Stefan, one of the devs has suggested I look into auto-correlation. I'll see if I can get my head around the theory some. Cheers!

BertAnt · 4 Jul 2009

Thanks for the compliment 🙂 That was in Reaktor, here's the link to it :

http://co.native-instruments.com/index. ... tchid=8409

I actually used a 'combo' technique, AMDF for note number and zero-crossing for pitchbend. The reason is because i found that for MIDI Note On events, the pitch decision must have a fast on/off switch character (just like keys in keyboard), so AMDF is great because it has high confidence between two adjacent notes, i could pick the same note repeatedly and it always output the same Note Number. But for pitchbend, i decide to go with zero-crossing because it's more like tracking than detecting (the pitch-wheel controller continuous character 😉) and that results in a fast and clean MIDI Note On triggering combined with expressive bends i think.

I definitely will need all your help since i'm totally new with SM 😀

I'll attach another MIDI file, which focused more on the bends 😉

Cheers !

oddson · 4 Jul 2009

I get the theory from that link. Stefan or Rudolf mentioned autocorrelation before in this context but I didn't grasp how it would work. I do now but I have no idea how you'd go about utilizing it as I don't think SynthMaker can deal with the matrix math involved (or at least I don't know how you'd do it).

BertAnt · 4 Jul 2009

Thanks for reply 🙂 It just using simple multi-tap delay, with the original audio get subtracted by each tap-delayed audio, so the AMDF equation is simplified a lot with this approach. After all, we don't need the precise pitch/freq from the AMDF, we just need the closest possible MIDI Note Number in short detection time. When Note number is known, Note On events can be triggered, then pitchbend can be tracked separately for getting on to the precise pitch/freq, with the original audio got band-pass filtered first (with the Note Number as the cutoff freq) before going on to the zero crossing counter.

I apologize that because i'm very new to SM, i might have problems describing what i have in mind, but this is very simple once you got the idea, much simpler than currently known autocorrelation algorithms, which i haven't grasp it completely yet.

Cheers !

BertAnt · 4 Jul 2009

There's simple experiment you can do to hear an interesting note 'muting', simply connect a delay, set delay time to match the period of the note you want to mute (f.i. A3=220 hz, set delay time to 1/220 sec), invert the phase of the delayed sound, then add the delayed (wet) to the original (dry) signal. Now try to play A3, and you'll only hear the attack but not the note itself 😉 The great part is it will still mutes A3 on, say, A3-C4-E4 chord.

This is what inspired me to use the multi-tap delay, basically any delay output with time that match the period of the note will be the quietest among others, so we can tell the pitch/freq of current note by the amplitude of the multi-tap delay outputs.

oddson · 4 Jul 2009

Ok... I've read the last few posts here with a little more care than I did late last night.

So... you need a tap whose delay is equal to the wavelength of the fundamental you're testing for. Add the inverted phase signal at the delay tap to the input signal and compare (?) to all the other tap differences to see which produced the greatest reduction in volume and assume that's the fundamental of the note.

Is that the basic idea?

(BTW, Nix - this, like zero-crossing is fundamentally and unavoidably a monophonic technique as would be proper autocorrelation.)

BertAnt · 4 Jul 2009

oddson wrote: Ok... I've read the last few posts here with a little more care than I did late last night.

So... you need a tap whose delay is equal to the wavelength of the fundamental you're testing for. Add the inverted phase signal at the delay tap to the input signal and compare (?) to all the other tap differences to see which produced the greatest reduction in volume and assume that's the fundamental of the note.

Is that the basic idea?

Yes, exactly ! 😀 It is much simpler than implementing the original equation isn't it ?

oddson · 4 Jul 2009

Well... the '(?)' in my post was meant to indicate 'how the heck to you do that quickly enough not to add excessively to the latency issue?'

The simplest way would be an envelope follower -- whichever output had the lowest env follower output over a certain window of time wins. But that would be slow.

You could take the difference signal over a single wavelength and calculate some average variance from zero and compare with all the others. But that would be complex and computationally expensive.

Another issue is octave range. The delay for C3 and C4 will both have roughly equal average values if the note is C3 but there's nothing to guarantee C4's won't be slightly lower. So you need to interpret the results so that the lowest note that produces enough cancellation wins even if an octave value higher gives a modestly better score.

Finally - pitch detection is only part of the battle, you also have to recognize note-on/off events including velocity etc. and the program would also be deaf to note bends.

nix · 5 Jul 2009

OK, I have Reaktor, so I was able to play your device. Man, it looks pretty complicated, but I'm up for trying to implement in SM. I noticed that it occasionally misses notes all together? Have I the settings wrong? Also I noticed how clean note on/offs are. I hope we can approach that in SM. The latency isn't too bad I thought. Anyway if you're keen I'll help if I can. I still am yet to understand your method though, sorry. I'm curious of this stuff. Anyway, maybe Barak could help, he has done SM conversion from Reaktor before. I hope you have fun with SM, that learning isn't too frustrating 🙂

BertAnt · 5 Jul 2009

oddson wrote: Well... the '(?)' in my post was meant to indicate 'how the heck to you do that quickly enough not to add excessively to the latency issue?'

The simplest way would be an envelope follower -- whichever output had the lowest env follower output over a certain window of time wins. But that would be slow..

Yes, you're right, i used env follower with zero attack time, the volume drops will be fast enough for determining the note quickly.

oddson wrote: ...You could take the difference signal over a single wavelength and calculate some average variance from zero and compare with all the others. But that would be complex and computationally expensive.

Well, i don't know if this is the 'correct' way of doing it, but all i do is just simply 'tag' each output with a Note Number value, and let the output with the lowest volume go through carrying the 'tag' while blocking the others, then checking the 'tag' to get the correct Note Number.

oddson wrote: ...Another issue is octave range. The delay for C3 and C4 will both have roughly equal average values if the note is C3 but there's nothing to guarantee C4's won't be slightly lower. So you need to interpret the results so that the lowest note that produces enough cancellation wins even if an octave value higher gives a modestly better score.

Yes, the well-known octave jumping between the correct note and the note one octave below will happen undoubtly. But, since i separate the measurement for each octave, the result can be simply determined from 4 or 5 note number value that represent the 'winners' from each octave measurements.

oddson wrote: ...Finally - pitch detection is only part of the battle, you also have to recognize note-on/off events including velocity etc. and the program would also be deaf to note bends.

Yes, from what i've seen pitch to MIDI is not just about pitch detection, but also trigger detection and the MIDI events transmitting as well. But with this approach, i found that the pitch and trigger data is easier to interpret for making MIDI events, because the switch-like character from the 'improper' AMDF with the multi-tap delay really represents each Note Number, like keys in MIDI keyboard.

nix wrote: OK, I have Reaktor, so I was able to play your device. Man, it looks pretty complicated, but I'm up for trying to implement in SM. I noticed that it occasionally misses notes all together? Have I the settings wrong? Also I noticed how clean note on/offs are. I hope we can approach that in SM. The latency isn't too bad I thought. Anyway if you're keen I'll help if I can. I still am yet to understand your method though, sorry. I'm curious of this stuff. Anyway, maybe Barak could help, he has done SM conversion from Reaktor before. I hope you have fun with SM, that learning isn't too frustrating 🙂

That would be great then ! I'm happy to share my other device, i already made a variation from the one in the link i posted. Yes, my pitch tracker relies heavily on the trigger, so if the trigger is not detected correctly then either it will transmits wrong notes or misses the note completely. The main idea is : when a potentially trigger signal is received, it search values from AMDF macros within short time for Note Number, then sends MIDI Note On message out, while also uses the same value as a cuttoff freq for the band-pass filter before zero-crossing counter, which tracks the pitchbend.

On the other hand though, i also has made the 'reverse', which relies on the pitch rather than the trigger, here :

http://co.native-instruments.com/index. ... tchid=8611

The reason because for tracking vocal it's much more comfortable if the pitch is the one that do the decision for MIDI Note Ons rather than trigger, at least from what i've found.

Also, i'm glad to send you the development stages, i'll upload it and post the link here, if you like.

And of course i can't wait to enjoy SM too, after i understand things a little bit, which i'm still not getting any at all..

Cheers ! 🙂

nix · 5 Jul 2009

Thanks for sharing! I'll play some more and go over what's been said and try to pick up. I'll dig a bit deeper and come back with questions, so maybe I learn a little Reaktor if you have the patience 🙂. Also I'll up the latest version of 2SYN and a couple of audio demos.

oddson · 5 Jul 2009

So here is something I put together earlier today but got the envelope follower low-pass wrong and so thought it didn't work as well as it does.

It barely scratches the surface of all the issues but it might help you guys. I think a fractional delay might be needed but to make one with so many taps was more than I was up for when I started on this.

Hope this helps.

nix · 6 Jul 2009

Thanks oddson! I think BertAnt is your man though. I'm having trouble following this 😊 . I'll have to go over it again. Much regard

BertAnt · 6 Jul 2009

I've been trying out modding several code, including the tap delay from oddson and analog kit, and now i'm very sure that this can be implemented perfectly in SM 🙂 Is there any tutorial for SM code ? I'm thinking of making separate codes for each function (delay, env follower, etc) and maybe connected them just like i did with my device. Also, sorry for this noobies question 😊 but how can i make if-then statement with SM code ? I read in manual it has something do to with masking, and i don't understand it at all. And in SM code, is the code line executed simultaneously for each sample or is it executed sequentially ?

Thanks and regards, Bert

nix · 6 Jul 2009

Is the info in the manual the same as the wiki?-> [old link] That's great you can do the port! I look forward to playing it 🙂

BertAnt · 6 Jul 2009

nix wrote: Is the info in the manual the same as the wiki?-> [old link] That's great you can do the port! I look forward to playing it 🙂

Thanks ! I'm not familiar yet with SM wiki, so i missed it 😊 Yet, i still don't know how long does it take for me to be able to port it 🙂 The quickest way i see so far is trying to emulate the modules i used with SM codes, although this is probably not the best way of doing it..

nix · 6 Jul 2009

I'll see if I can grasp any part of it to try port with primitives, as I haven't yet learned code Cheers

BertAnt · 6 Jul 2009

Great ! I'll try to break down the device's structure into more distinct parts than currently, then upload on Reaktor UL and post the link here 🙂

I just notice the thread title has changed 🙂

Cheers !

BertAnt · 7 Jul 2009

Ok, here's my first attempt to port the pitch detection part, first of all sorry for lots of noobies mistake 🙂 , it takes four hours for me to translate just this section.. whew. But i have troubles testing it, i can't make ASIO In-Out working, always crashes SM when i try to activate audio in and out together 🙁 so i don't know how to include it in the osm.

Regards, Bert

edit : I don't understand how the Pitch to Frequency converter works, seems like it doesn't map pitch value to related frequency ? Or maybe i'm making mistake in using it ? Also, how can i make streams active so the value can be read on display ?

oddson · 7 Jul 2009

BertAnt wrote: ... edit : I don't understand how the Pitch to Frequency converter works, seems like it doesn't map pitch value to related frequency ? Or maybe i'm making mistake in using it ?...

Are you familiar with the two frequency representation systems used in S|M? [old link]

The Hertz based one is meant to make it more comprehensible for someone working with only high-level modules but it can be confusing for noobs who start right in with the gory details.

BertAnt wrote: ... Also, how can i make streams active so the value can be read on display ?

To read stream values type 'read' in the toolbox filter and you'll find poly read and mono read modules that sample the stream and display the results.

BertAnt · 8 Jul 2009

Thanks oddson ! Sorry for not knowing this two representation, i've never heard of normalized freq before, and after reading the wiki i now know there's a module that can converts normalized freq to hertz. And what i mean with reading streams value was how to turn on / off streams that goes through the module so its output can be read on mono /poly read. But i figured out a way, don't know if this is right though, connecting to ASIO In and Out then clicking the drivers tab to turn on / off streams.

Nix, how's it going with the porting ? Anyway i just put a macro that contains only the pitch detection part here :

http://co.native-instruments.com/index. ... tchid=8628

This is the main part from my device, while the rest is just for preparing the raw data for MIDI Note On / Off transmission.

Regards, Bert

MegaHurtz · 8 Jul 2009

If I understand correctly you are not looking for the zoro crossing method? The crossing is quite easy.

BertAnt · 9 Jul 2009

MegaHurtz wrote: If I understand correctly you are not looking for the zoro crossing method? The crossing is quite easy.

Yes, zero-crossing is simple to implement, but i found it's hard to interpret as MIDI Note Numbers (especially for guitars since the attack of the string can lead to unexpected and jumpy results) . After struggling with it for a year i decided to look for alternative way, then i found out about autocorrelation, AMDF, and phase cancellation phenomenon. And i was very impressed with WIDI mono correlator mode, which made me believe this approach is very promising for a monophonic pitch to MIDI converter.

BertAnt · 11 Jul 2009

I think i'm making some progress, here's the second try. Now it can detect pitch within 1 octave range.

Bert

edit : I've replaced the previous with the one i've just worked on. Optimized the code a little bit for lower CPU usage and added MIDI Out.

BertAnt · 17 Jul 2009

Ok, now the Note On detection is pretty much ported quite well. But seems that the latency is very high on some systems, i hope i could get feedbacks about it.

Thanks, Bert

nix · 18 Jul 2009

I'm sorry Bert, i couldn't get your VST working in Orion. If you added a midi out select module maybe? Tell me what I'm doing wrong, I stuck it on an audio in channel in the mixer and set a synth to receive midi from that channel. Sorry I haven't replied here for a while 😊 . I'll see if I can get your latest .osm up. Much Regard

nix · 18 Jul 2009

Woah man !!! This works great 🙂. I added a delay before the gate and it's tracks so sweet with guitar->

I'll see if I can make it track the fretboard, although you have already done. You should sell this man. Cheers and thanks for the great device Sorry for not replying so long I'll make an audio demo if I can get it track fretboard

BertAnt · 18 Jul 2009

nix wrote: I'm sorry Bert, i couldn't get your VST working in Orion. If you added a midi out select module maybe? Tell me what I'm doing wrong, I stuck it on an audio in channel in the mixer and set a synth to receive midi from that channel. Sorry I haven't replied here for a while 😊 . I'll see if I can get your latest .osm up. Much Regard

That's ok, eventually i'm getting used to work alone in Reaktor 🙂 Yes, i should add MIDI Out select module, i forgot that because in Cubase SX3, MIDI out from the plugin shows up on the available ports..

nix wrote: Woah man !!! This works great 🙂. I added a delay before the gate and it's tracks so sweet with guitar... ... I'll see if I can make it track the fretboard, although you have already done. You should sell this man. Cheers and thanks for the great device Sorry for not replying so long I'll make an audio demo if I can get it track fretboard

Thanks ! 🙂 The gate in the osm still not synced yet with the pitch detection, i haven't completed the combiner module that merges data from 4 one octave detector and generate gate based on the combined data when i uploaded the osm, that's why the right pitch is still not guaranteed when note on was generated.

If you want to figure out how to track the whole fretboard, i'll hold the latest osm for a while, but if you want to know right away i'll happily post it 😉

About the selling, well.. i just want to do something to the dying audio - MIDI converter world, i don't like the fact that most software company only concentrated on wave file conversion, not realtime one. There's a big difference when you could actually play the synth than record a wave first then converting it to MIDI, even if the non-realtime conversion is perfect. And i just want to make something that not only tracks good, but also feels good too on guitar 😀

For any update to the plugin you can check here :

http://www.box.net/shared/7u4703b92y

Regards, Bert

nix · 18 Jul 2009

I would really appreciate your .osm!, I think it has great potential. I'll keep thinking about polyphony too, but velocity would be more realistic. It plays really nicely the 1 octave version I have. I'll check your latest .dll. Cheers!

BertAnt · 18 Jul 2009

Ok Nix, here's two version of the .osm, the first one still divides each octave tracking into separate modules, the other is a tryout using single 4Mono 12 tap delay and lots of bus experiment 🙂 And i have simplified the test synth within it, now the envelope is taken from the audio input's envelope.

Polyphony tracking probably could be done with FFT module, i'm interested to try it too. I'll post the ported velocity extractor when it's done 😉

nix · 19 Jul 2009

Thanks heaps bert, I'll experiment with what you've done to try and get the best tracking. It sounded cool with my analog synth when I set it to make 1 octave apart 2 note chords. Cheers!

nix · 21 Jul 2009

Here is a short demo with the version I am using-> http://www.sendspace.com/file/aoqjq1 Is it alright to share the.osm bert? I will make a GUI Warm regards!

BertAnt · 21 Jul 2009

Thanks for making the demo Nix ! I guess i should stated more clearly in KVR that i've already made a demo (it was in the zip file), i'm sorry that you got 'rough' comment about the one you made..

Of course it's ok if you want to share the .osm, feel free to do it ! 🙂 Also if you like to do the GUI, go ahead, although i think i might add more functions and changing several thing, check the last one in box.net link i gave.

Regards, Bert

nix · 22 Jul 2009

Oh OK 🙂. I'll have a listen. That demo I made actually sounds nice and warm(fat!) when pitch shifted down an octave, I'll work on it, I did a little more recording. No worries! I like this a lot. It's a different feel from hexaphonic. There seems to be an intermittent problem with one note per octave. It plays the same note on two different frets most times, although sometimes it's right. Only on those 2 notes per octave. I tell you which note today. I think C.

One function I have found useful (musical!)is auto chording, power chords and + 5 semitone. Is it possible to make it so that instead of muting the last note played when another note still sustaining, could you mute the prior note, so it's last note played sounds? That would be great for me.

I will up here the .osm with GUI soon 🙂

BertAnt · 22 Jul 2009

That's why i feel so bad about it, if someone has to received bad comments i'm the one who should, not you 🙁

Thanks for the report 🙂 Yes, it's very sensitive on some fret/string i guess, on my guitar it won't play E on 12th fret, but i think it's mostly because i use old rusty strings and i don't tune my guitar correctly 🙂 But hopefully when pitchbend is done, this won't be a problem anymore. The difference from hex is because the note on decision and 'rules' for transmitting it i guess, i want something that very stable for playing percussion 🙂

Sorry if i don't get it correctly, but you can use damper/sustain pedal for chord building, correct ? And creating harmony like 5th is not much a problem i think, i can add more MIDI event primitives for it and more transpose options 😉 Also about the sequence of muting, it's because of the 'rules', i'll try including more options for it.

Looking forward to the .osm with GUI !

BertAnt · 24 Jul 2009

Here it is Nix, finally got velocity and pitchbend working 😀 But the pitchbend is still not sounding good, needs more tweaking i guess.

nix · 24 Jul 2009

Great man! I'll try tonight. I think I'll be recording a lot more stuff with this, so. 🙂 I'm working on a project for the DC so not full time on GUI, but I'll send you a knob I made but haven't used to see if you like it.

BertAnt · 26 Jul 2009

I'll be waiting for your comments 🙂 Here's a fun MP3 i made 😉

BertAnt · 15 Aug 2009

Ver 1.34, in case anyone interested 🙂