Sonification of Arsenic
Idea behind Sonification
« the use of non-speech audio to convey information or perceptualize data. Auditory perception has advantages in temportal, spatial, amplitude and frequency resolution that open possibilities as an alternative or complement to visualization techniques. ». from the wikipedia
It can also be simply seen as an interface between the human and the machine.
The Aim of the Sonification of Arsenic was to improve the actual prototype to make it more intuitive.
The basic concept is that the bacteria are producing light which is converted into frequency thanks to TSL235 (a light to frequency converter).
This frequency could be used to perform some specific sounds depending on its value. We wanted to play a music while counting, to inform the user the measure is being taken, that it is processing.
And finally a specific music that will become more and more dramatic the higher the concentration of arsenic the sample is containing.
We worked with two different prototypes:
- Working with a Piezo Speaker
- Working with a Music Shield
At the 5th week of the semester, we started our work on Arsenic sonification.
We learnt about Arduino and understood that it was the central element of our circuit. We found several tutorials with Arduino producing sound and most of them involved a piezo speaker.
Our first circuit was simply an Arduino and the piezo connected to each other:
We discovered how to code for different sounds and how to build melodies. (code on github)
Now to adapt this to the prototype, we added the Light-To-Frequency (LTF) converter to the circuit. Indeed this component is the one that gives the information about Arsenic concentration in the prototype.
How does it work ? it detects light and turns it into a square frequency :
We found on the web an example of a code using the TSL235 LTF: The Arduino is counting the number of “counts”: that is the number of times the frequency goes from one to zero. We modified this code so that over a specific threshold of counts, a different note is being played. (Link to the code)
In order to get closer to the prototype, we then tried to adapt the code for the piezo to the code of the prototype. The Arduino code previously developed used two files: Counter.cpp and Counter.hpp.
The difficult part was to understand the timing of the program: the Arduino took a specific delay for the measurement and the counting was done repeatedly. What we wanted to achieve was playing music when counting and then playing a specific music : stressful if highly concentrated in Arsenic and normal if low concentration of Arsenic. We chose some melodies found on the web.
Here is the circuit with the LTF:
And the link to the code.
The next idea was to use an Arduino shield to produce some real music.
However, this part was quite complicated to achieve therefore we continued with the Piezo Speaker to create a Music box (github) to present at the STIL. To create the music box, we started from the same circuit and code but we added some components to add some dynamism to the prototype.
Prototype Music Box
The idea behind this circuit was to model the detection of the fluorescence emitted by the bacteria: if we apply an intense source of light directly to the light-to-frequency converter (which is the same than the one used in the prototype), it models a great intensity of green fluorescence (and thus supposedly a large concentration of Arsenic in the sample). If no light is applied, the light-to-frequency converter only senses ambient light which models the situation when the concentration of Arsenic is so low in the sample that not much fluorescence is emitted by the bacteria.
The basic components of this circuit are the light-to-frequency converter (TSL235), a piezo speaker (to produce sound) and the arduino manage the input and output of the components. We also added a RGB LED to keep visual information, two potentiometers to modulate the sound produced by the piezo (one for the volume and one for the speed) and two buttons to be able to choose two different musics.
The LFT sends a squared frequency signal to the arduino. The file Counter.cpp counts the number of time the signal goes from 0 to 1 and it is this number of counts we compare to a threshold to determine if the light intensity (imaginary arsenic concentration) is acceptable or not. Our prototype shows here a limit because the threshold has to be adapted to the light intensity of the environment where the device is being used.
Here is the scheme of the circuit:
In order to finalize our prototype and our project, we decided to solder the components of the Music box on a printed circuit.
We then designed our printed circuit board using the EAGLE (Easily Applicable Graphical Layout Editor) software, which has a freeware version available. To learn how to use this software, we found a tutorial on Sparkfun. We added two different libraries on EAGLE in order to find our specific components. Both libraries are on github, the first one is from Adafruit, and the second one is from Sparkfun.
Components of the circuit:
- arduino uno
- 2 push buttons
- 1 RGB LED
- 1 piezo speaker
- 5 resistors
The design of the a printed circuit on EAGLE happens in two steps :
First we created a schematic (once we found all the components, we connected them on the Schematic following the fritzing scheme) :
Figure of the Schematic of the Music Box
Then we designed a board from the schematic. The board is not created automatically: you have to dispose the components yourself so that no line crosses each other, because this will lead to a short circuit. We also had to be carefull on which side of the plate the components would be solder and therefore do the routes on the right side. That took us quite some time to achieve. Especially we had to design on Eagle the potentiometers because we couldn’t find the specific type that we had. We also modified the spots of the RGB led because they were too close from each other.
Figure of the board of the Music Box
Once the design of the circuit was done, we printed both sides on two plastic films and we aligned them in order to have each on one side of the copper plate. The printing has to be done quite precisely because every mistake made by the printer is found afterwards on the PCB. This is also why we used two films for each side : the printer makes small random holes in the black area of our circuit (only seen with the microscope naturally) so if we add another film the random holes won’t be at the same place thus the result will be more precise. However in order to achieve this precision the two films had to be perfectly aligned with each other.
Then, we went to the laboratory and followed a specific procedure:
- Wash the copper plate with water and alcohol to remove fats. We had to scrub the plate quite hard because the fat makes bad conduction so we want to remove all of it.
- Dry the plate in the oven at 80°C.
- Cover the plate with photoresist with this machine. The plate goes down in the tank and is removed automatically.
- Then put the plate in the oven at 80°C for the photoresist to dry (8 minutes)
- At this step we actually print the circuit on the plate: put the plastic films and the plate in the UV machine and the photoresist will go away only in the areas not covered by ink
- When we reveal the plate afterwards (with developer), the circuit is already distinguishable on the plate
- The plate is then put in the etching machine to eat away the remaining copper that is not covered by the circuit
- The final step is the stripping: the remaining photoresist is stripped away with alcohol, leaving only copper
We printed it twice in case something went wrong. After that, we checked with the microscope that the printing was correct, we drilled the holes and we cut the circuit to have the following final result:
Drilling the holes and cutting the plate
Finally, the last step was to solder the components on the PCB but we didn’t finish that yet.
Final current version of the Music Box
Working with a Shield
After having worked with a piezo speaker, we wanted to hear some real music and not just synthesized one. Our idea was to use a Music Shield v2.0 which is a device that can be plugged on Arduino and that plays real music. In order to achieve that, you have to insert a micro SD card with music files on it and to connect the shield to a speaker. It is very easy to use and quite cheap to produce. More details on music shield from seeed studio
We first learnt how to use the Shield on its own. We added a library to the libraries of the Arduino software. We found it on github from seeed studio
We simply used the code given as an example in the library of the Music Shield:
File -> Examples -> MusicPlayer -> createList
Then we adapted the code to be closer to the prototype and created a circuit with the light-to-frequency converter (LTF) which is used to detect Arsenic concentration. (<a href="https://github.com/biodesign-coco/Sonification/tree/son-counter">Link to the code</a>)
The first problem we encountered was that it only counted light intensity once; then the music stopped and it also stopped counting. It seemed to us that there was no link between the music played and the counter. Then we learnt that the problem was linked to the TIMER: the Music Shield we were using used the TIMER1 of the arduino that was already used to count the output of the LTF.
After a discussion with Robin Scheibler, he proposed us three different options to solve this problem:
- Use an interrupt to measure the output of the LTF. (Link to the code)
- Modify the music shield library to use TIMER2 instead of TIMER1.
- Use two arduinos: one for the music and the other to measure light. (Link to the code)
With a Counter
We tried the first option by adapting our previous code with the code of interruptCounter. The advantage of Interrupt is that: «Interrupts are useful for making things happen automatically in microcontroller programs, and can help solve timing problems. » (Arduino.cc). To better understand how attachInterrupt worked, we found some reference on the arduino site.
But this solution did not solve our problem: the songs were often not found on the SD card and it seemed that there was still no link between the music being played and the counting as if both were acting completely independently. And the program was finally always interrupted in a random way.
With an Interrupt
We decided that this solution was not the right one, thus we decided to work on the second one. We tried to modify the library of the MusicPlayer to put the arduino shield on TIMER2 (instead of TIMER1). This option revealed to be too complicated for us since we did not understand all the methods related to the timer and how exactly we had to change them. The documentation found on the web about this was also quite limited.
With two Arduinos
Therefore the only solution left was the third one: to use two different arduinos. This way there would be no more problem with the timers because each arduino has a TIMER1. The communication between the two arduino consist of one being a Master, which gives orders to the other, the Slave. We decided that the Master had to be the one with the LTF, and the Slave the Music Shield.
The first problem was the fact that we only used a unidirectional transmission. The master was telling the slave to play music, but just went on counting without checking if the song had been played entirely. The result was that it switched between the musics quite rapidly without being completely played.
We came up with two solutions: the first one (the easiest one) was to put a delay so that the music had time to be played. This solution worked well but didn't allow the modulation of the code: the delays would have to be adapted to each song we want to play since they don't have the same duration. The second solution, which seemed more accurate, was to create a bidirectional communication between the two arduinos:
- Transmit to Slave when to play a specific music
- Request from Slave when the song is over
- Receive from Master when to play a specific music.
- Send to Master when the song is over.
To determine when the song was over, we had to add a method in the class of the MusicPlayer library. This method returns a byte if the song has been totally played and returns 0 byte if it has not.
After many trials and fails we finally managed to produce a reliable connection between the counter and the shield. They are both considering the state of the other before going further. The Counter is waiting for the Music shield to play the entire song, and the Music Shield is waiting for the Counter before playing a new song and to choose which song to play.
The next step will be to adapt this code to the prototype. Now that there is a clear working interaction between the two arduinos, it will become easier to continue.
Comment: only the code with the two arduinos is doing what was expected.
Future direction possible:
Speaking with Marc Dusseiller from hackteria, he proposed us to use the mozzi library for sound synthesis and also playing samples. We did not have the opportunity to try it since we were blocked on the timer issue for a while. And this library would not have helped us when we had trouble since it also use TIMER1.
Sonification Future Directions and Conclusion
Finally we managed to create a circuit that produces music depending on light intensity and can be further adapted to depend on arsenic concentration.
We had multiple problems which prevented us from achieving all of what we had planned. But we finally solved them and we finished with two fonctional prototypes: a Music Box, with a piezo speaker, and another one with two arduinos and a music shield, which produces real music.
The future directions of this part of the project would be:
- use the Mozzi library for the Shield
- Script to adapt the code to any data
Depending on the country, there is not the same concentration allowed (i.e 10 µg/L for Switzerland, 50 µg/L for India). Creating a script to adapt the code depending on the country where it is tested (or other factors that fluctuate with the environment) would make the prototype more accurate.
- Statistical analysis of the measurements
Performing a statistical analysis on the collected data would also improve the accuracy of the measurement.
- Get rid of the computer
To make the prototype even more hand-held and accessible, we wondered if it would be possible to get rid off the computer (needed for arduino) by replacing it by a smartphone or an adapter that would supply power.