Intrinsic Jazz

Intrinsic Jazz is a VR training tool for elementary qualities in jazz such as beat tracking, chord tracking, and solo line forming.

[Introduction]

Jazz brings a mix of improvisation, creative ideas, and collaboration among musicians. Underneath such interplay of music lies a carefully woven pact of various musical features such as beats, rhythm, chord progressions, and original melody lines. These often need to be intrinsically embedded in the performers in order to spontaneously pursue a creative musical network. Intrinsic Jazz aims to train players of different experience levels to further internalize fundamental elements in jazz such as beat tracking, chord tracking, and solo line forming.

[Skills]

Beat Tracking: Which beat am I on? Am I on the stronger beat? Can I detect the first beat of the measure?
Chord Tracking: Which measure am I on? Which chord am I playing? What is the relationship to the previous chord?
Solo Ideas: Which notes should I choose? Should I make it motivic? Should I develop the idea further?

[Pre-gameplay (PC)]

Narrative: One day, to escape from the mundane, you decide to make a visit to a shabby jazz cafe. While waiting for the gig to play, the manager hastily snatches your wrist to the backstage and says, “Our pianist isn’t showing up tonight… You must save us. Today may be the last night for us! Please be our pianist tonight. I can teach you real quick!”

Tutorial #1 – Beat Tracking: Visual metronome is shown in the beginning and whenever the wrong beat is clicked. The player listens to the drum loop of the selected difficulty. The player’s task is to click on the bubble every first beat of the measure.

Tutorial #2 – Chord Tracking: The chord sequence of the selected difficulty is shown in a sequence. The player listens to the entire music of the selected difficulty. The chords will be shuffled and the player has to rearrange them in the correct order.

Tutorial #3 – Solo Line Forming: Chord tones will be shown according to corresponding chords in identical colors. The player listens to the entire music of the selected difficulty. The player’s task is to sort the right notes to each measure according to its chord.

[Gameplay (VR)]

The player enters the VR space and plays the song of selected difficulty. Player hears drums and bass. If the player does the task well, piano sound will be played.

  1. Melody: The player pops the right chord bubbles on the first beat of every measure. Both piano chords and melody will be played.
  2. Bass Solo: The player pops the right chord bubbles on the first beat of every measure. Only piano chords will be played.
  3. Piano Solo: The player pops the right note bubbles on the first beat of every measure. Solo lines will be played.
  4. Melody: The player pops the right chord bubbles on the first beat of every measure. Both piano chords and melody will be played.

[Neuroplasticity]

Limb et. al. (2008) states that jazz musicians playing simple scales based on chords and playing complex improvisation showed similar activation and deactivation points in the brain. This bolsters our simple solo line forming experience (which is close to scale improvisation) hold similar neuroplasticity to actual jazz improvisations.

Also the portion of MPFC that is activated during improvisation appear to serve a broad-based integrative function, which means that multiple cognitive functions are combined to achieve higher behavioral goals. This game breaks down those multiple cognitive functions into pieces and train them separately, which would aid stimulating the improvisation-activated areas.

Bianco et al. (2017) conducted a study that revealed jazz musicians have different neural responses than classical musicians to harmonic processing. This shows that there is a unique receptive area for neuroplasticity to occur. Thus jazz training has exclusive characteristics, and this game is aiming to achieve them.

Mitchel et. al. (2014) stats that learners of their experiment were able to extract multiple statistical regularities across modalities when there is some degree of cross-modal coherence. This game heavily depends on both audio and visual in performing the tasks, which would enhance players’ statistical learning that otherwise cannot be granted by either mode alone.

[Reference]

– Limb, C. J., & Braun, A. R. (2008). Neural substrates of spontaneous musical performance: an FMRI study of jazz improvisation. PLoS one, 3(2), e1679.
– Bianco, R., Novembre, G., Keller, P. E., Villringer, A., & Sammler, D. (2017). Musical genre-dependent behavioral and EEG signatures of action planning. A comparison between classical and jazz pianists. NeuroImage.
– 
Mitchel, A. D., & Weiss, D. j. (2011). Learning across senses: cross-modal effects in multisensory statistical learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1081.