It’s been four months since the previous article was written, and I have a lot of news both in the technical part and in the positioning of the device.
Let me remind you of the requirements I set for the device when I started its development:
- It has to imitate a 6-string 12-fret guitar.
- It has to be quite small, ideally collapsible, for a person to carry it anywhere.
- It has to support all popular operating systems — Android, IOS, Windows, Linux, MacOS — and be identified as a MIDI device without any drivers.
- It has to support the power supply.
- It has to support wireless Bluetooth Low Energy connection (but since it will have a USB port for charging, it will be possible to connect it using a cable)
- It has to have the opportunity to start playing right away, without any need for training and adapting your hands.
- Each string and element of the fingerboard has to have a LED to start the tablature of the tune, and the guitar itself should show where to put your hands.
- There has to be an opportunity to use the main guitar playing techniques: hummer on, pull off, slide, vibrato.
- Latency of carrying MIDI event messages has to take not more than 10ms.
- Everything should be made out of simple elements without any difficult technical processes or expensive electronics.
I’ve managed to do all of this and even more. I’ve also integrated an accelerometer to manage the parameters of using digital filters by tilting the guitar and the vibratory motor (I still don’t know why, though).
While I was writing the previous article, it looked like this:
I’ve made the decision to try launching a startup and use Kickstarter.
So, what happened next?
The next step was to find a pre-seed investment round. I needed money to improve and develop a new prototype, engage in promotion, and pay for legal processes (to use Kickstarter, you need to have a legal entity in the USA, so you have to deal with all the organizational expenditures). My partner (who is responsible for the business processes) and I managed to solve this problem in two months.
Positioning
To confirm the product’s usefulness and functionality, we decided to meet all the musicians from Saint Petersburg and Moscow that we could contact; we then filmed videos for social media.
Professional guitarists were quite skeptical of the device. I totally understand them — you can’t replace a real guitar with it. This device should be used in cases when you need portability, universalism, simplicity, opportunity to use it wearing headphones, etc. Imagine: you are on plane, children around you are crying, it’s very noisy, emergency oxygen masks are dropping. You can’t sleep. Suddenly you feel inspired, take the device, put on your headphones, and record a new track.
The general public really liked the guitar despite the limited functionality and big constructional disadvantages of the current prototype. The device turned out to be so fascinating it even got reviewed by a popular Russian blogger, Vaganych.
After the discussion, we realized what our target audience looks like — sound designers, music makers — people who deal with digital music. It turned out that they have serious difficulties with recording guitar and other string parts using MIDI keyboards. It’s really complicated and takes so much time that sometimes they have to give up using string instruments, and that limits their art. They were super enthusiastic about our device and said it’s going to be really popular.
Rivals
We have rivals in this market, so we’ve bought their products for review.
-
Artiphon – a touch-sensitive pad shaped like a guitar but positioned more like a MIDI-controller.
It’s an interesting but expensive product. In general, it works well, and you can play different instruments using it. It has built-in speakers, but it’s better not to use them. It can be connected only via USB.
- Jammy – a guitar form-factor. It consists of real strings and elements that can be disassembled. Sensors independently track the string tapping as well as how the strings touch the fret. The guitarists I know failed to play anything well using it; the notes either didn’t play at all or played twice. Maybe it requires getting familiar with; and it takes time. I also watched some video reviews and couldn’t find any real guitar playing, only slow fingerpicking.
Analyzing our rivals made us more optimistic. Neither of these devices allows people to play fast guitar parts or have a training backlight.
Now let’s talk about the most fascinating part — our new prototype
Based on the experiments of the old prototype, rivals’ products, and my own preferences, I decided to broaden the functionality of the device and add several requirements:
- RGB backlight.
- Soundboard strings with touch-detection and detection of the deflection degree while playing the notes. It would solve the problem of touching other strings by mistake and would broaden playing opportunities thanks to velocity (different parameters of making the sound and the note volume louder) and string muting by touching, like on a real guitar.
- Detection of the pressure level over the sensors on the fingerboard to use standard guitar playing techniques.
- Built-in synthesizer with different instruments and a Jack 3.5mm port to plug in headphones or external speakers. I decided not to make built-in speakers — quality sound would have cost too much and would have been too complicated to achieve.
- A mobile app with a built-in qualitative synthesizer and learning functions.
- Pads with backlight to record loops.
- Special shell on the soundboard to keep it conveniently both if you are sitting or standing, strap locks.
Body
The body got a lot of changes, especially in the string assembly on the soundboard and the folding mechanism. The sensors on the fingerboard are now covered with an opaque diffuser, which lights up over the entire surface.
The main challenge was making the device so that it could be used as a real guitar. I’ve worked on the distance between the frets, strings, soundboard geometry, and weight distribution. These are only body renders for now, but we are already producing the prototypes.
Electronics
I had to divide electronics into four motherboards:
- Fingerboard
Addressable RGB LEDs are connected in sequence to one of the STM pins. I had quite a job with double buffering and data output via DMA 100 times in a second, but now it works really fast, and you can launch colorful visual effects on the fingerboard.
- The motherboard with pads and spring-loaded pins to connect the fingerboard when the guitar is fully assembled
- The main motherboard with strings, brain, power part, radio part, synthesizer, and sensors
That is where the touch-detection and detection of their deviation take place. It solves the problem of accidentally touching another string while playing and gives the opportunity to damp strings like on a real guitar.
-
The motherboard with USB type-C and Jack 3.5mm ports and three indicator LEDs
Mobile application
After posting the previous article, I got a message from a mobile app developer, who suggested help with the iOS app development.
We’ve made a protocol for communicating between the guitar and the app and divided it into several interfaces:
- MIDI event messages are carried via the standard BLE-MIDI interface by default. Thus, if you need to connect other MIDI devices (e.g., a keyboard), you can.
- MIDI event messages can also be received via a USB-MIDI cable interface. It will be useful for old phones which don’t support BLE and also in case you need to minimize the latency (about 5ms)
- A separate custom-made BLE service for sending unique event messages about LED control, device operation modes, state synchronization, etc.
- Standard BLE battery service to show the battery level. It’s supported on the operating system level, and if we speak about Windows, you can see the icon on the device panel.
The app is divided into several screens which correspond to different work modes:
Free playing
This mode lets the user choose one of the instruments (an acoustic guitar, an electric guitar, a piano, a ukulele, drums, etc.) and play just the way they want to. There is an opportunity to upload your own instruments in the form of sound fonts as “.sf2” files.
Tablature playing
Here you need to choose a tune out of the tablature list and run the interactive playback. The guitar uses the LEDs to light up the chord you need, and the app waits for the user to play it; only then will it light up the next chord. While doing this, it evaluates the accuracy and the speed of play.
Training
We don’t even have a prototype of this mode yet, but it is supposed to be interactive learning of musical notation and guitar basics through play using the LEDs on the device.
There are also other interesting modes which I’m going to talk about in the next article. We still have a lot of work to do.
At the moment, you can play it three different ways:
- BLE MIDI protocol connection to the phone or computer where the device is identified as a MIDI device and you can play using external virtual musical keyboards (Ableton, FL Studio, Garage Band, etc. or our app).
- Everything is the same except for the connection — here it is USB MIDI (works with all hosts I checked — Android, IOS, Windows, MacOS, Debian).
- If you are playing the built-in synthesizer with the connection of headphones or an external speaker directly to the guitar, the sound won’t be that Hi-Fi but it will be quite good for individual usage.
By the way, you can also use all three variants of playing simultaneously, which may be helpful.
Completion
Once I complete body manufacturing and testing of a new prototype, I’ll write the next article to talk more about the technical details. If you want to take part in content making, share your ideas or help with marketing — please, write to me. We would also be happy to get help with body manufacturing and app development.
If you want to receive news about our project or place a pre-order — leave your e-mail on the Sensy site and subscribe to our social media. Please, help us to promote our startup, this will significantly speed up the launch of the campaign and allow us to add more functionality.
Thanks for your attention! I’ll excited to read what you think about all this in the comments.
https://www.youtube.com/watch?v=3eGYQ3LV-Js&ab_channel=SensyGuitar