Neuromancer

When I was in the animation department at VCU, I had an idea for a project that was essentially to create an interface for a human being’s internal state of being.

I imaged some heart rate indicators, mood levels, brain activity graphs, and maybe a log of thoughts.

Purely 2D dashboards are boring, and I readily imagined a slow rotating 3D model at the center of it all. Many years later I finally got around to starting to build it

I enlisted my downstairs neighbor Mike, who offered the use of the photography studio that is his apartment. We used a free photogrammetry app called Polycam to take about 130 pictures which where then used to create a model of my head.

Photogrammetry is the process of using photographs from many angles to create a 3D model, often complete with textures.

I downloaded the model and created a website using ThreeJS Fiber. ThreeJS is a Javascript library for creating anything 3D in the browser you’d like, which I’ve used to create this game and this similiar project.

ThreeJS fiber is a way of using ThreeJS as if it was a set of re-useable React components. It felt strange at first, but in the end it was really nice to create those components like “Head” or “Pointer” with JSX that took parameters that would change how they were rendered.

interface PointyProps {
  surfacePoint: THREE.Vector3
  distance: number
  text: string
  link: string
}
 
const Pointy = ({ surfacePoint, distance = 0.5, text, link}: PointyProps) => {
    const transp = .2
    const pointyRef = useRef()
    const direction = surfacePoint.clone().sub(new THREE.Vector3(0, 0, 0)).normalize();
    const endPoint = surfacePoint.clone().add(direction.multiplyScalar(distance));
    
    return (
      <>
        <Line
        points={[surfacePoint, endPoint]}
        color="white"
        lineWidth={.7}
        transparent
        opacity={transp}
        ref = {pointyRef}
        />
        <Spherey position={[surfacePoint.x, surfacePoint.y, surfacePoint.z]}/>
        <Texty start={surfacePoint} end={endPoint} text={text} link={link} size={.02}/>
      </>
    )
}

Above is the component used for each of the tooltips sticking out of my head

The Result

So what did I end up creating? A weird, interactive “map” of my head! Please play around with the live version here.

It’s mobile friendly but if you’d rather watch a video of it, here you go:

It’s a sort of personal site. I’ve used some very rough neurological regions, plus some symbolic parts of my face, to “point to” different bodies of work I’ve created.

I still have yet to create and add all of those mindstate indicators. I also would like to create some sort of cutaway, or alternative layers, that allow a visitor to truly see inside of my brain.

3D Space as a ‘Better’ Canvas for Ideas

There is a deeper reason I chose to work on this project after all these years. That reason is that I don’t think the prevailing paradigm for information presentation on the web is the final one we will arrive at as a species, and I wanted to begin exploring rival paradigms. Namely, a spatial one.

The prevailing paradigm for most information is roughly 1 dimensional. Novels, websites, brochures, etc. The information is organized in a relatively structured pathway down the page. Line by line, going down, and then in English at least, left to right for each line. This generally includes images.

Columns and pages can be added, but these don’t really augment this basic paradigm too much in terms of dimensionality. An article written with multiple columns is still one-dimensional in that the reader can progress forwards or backwards, and there is no other vector of experience to navigate on.

I could argue that TV shows and Movies generally follow this same one-dimensional paradigm.

A newspaper is slightly different, with it’s different areas of information. Sports, Style, etc. The reader has a few more choices. However, they are still presented in a linear fashion for the most part, but the viewer can hop from article to article in a multidirectional way that we could call 2 dimensional.

A better example of a two-dimensional data presentation is a spreadsheet I think. A chart with two axis and information in a grid is a different type of data structure with no assumed order. Forward and back is no longer a meaningful distinction. You can increase or decrease in magnitude on either the X or Y dimension though, which is similar. What’s interesting though, is that different combinations of traversal on each of these dimensions produce different results. A graph is also an example of this. This dimensional offers more information in the same space, but it’s also more ambiguous. A user can navigate the dimension however they would like.

Then there’s three dimensional data structures. In a way, this is what most people are most familiar with. They don’t require any abstractions by default, as they can exist in the same way that our default three-dimensional world does.

I suppose that our default world is more like 4 dimensional, since we’re also moving through the dimension of time, but unlike the 3 dimensions before it, human beings (the small subsection of the ones I know at least) don’t have control of navigating the fourth dimension, and so it would make sense that doing so artificially in a data presentation would be abstract, as it’s not something we do ever. But navigating three dimensions, that’s something we do all day.

The third dimensions has some really special qualities. I think that one of them is most evident in 2-dimensional, infinite canvases, like Figma, Milanote, or Miro. These tools allow for information to be distributed flexible, and moved around like paper on a desk, across a surface. On the 2nd dimensional level, they have a really nice quality where they may be sort of default forward and backward movement between slides, but on a 2D canvas, some sides can be put to the side making use of that 2nd dimension to open up another space that is used for unorganized things, or things that maybe exist outside of some default continuity.

Something happens though, when a user zooms in and out of a 2D canvas. It activates depth that allows things to be clusters. It allows for the use of negative space to separate things - unlike concepts can be given more space, alike concepts can become neighbors. This use of zooming, which is really a 3rd dimensions, allows for scoping.

In our lives, this is a paradigm that comes up constantly. There are many human phrases that make use of this idea.

---

Detail Biased

• “See the forest for the trees”
• A lot of the things that Bob Ross says
• “Granular”
• “Leaving no stone unturned”
• “Fine-toothed comb”
• “Diving deep” ← this one is interesting because rather than detail, it references scope as an altitude
• “Needle in a hay stack”

Abstraction Biased

• “Seeing the bigger picture”.
• “Sweating the small stuff”
• “Tunnel vision”
• “Losing perspective”
• “Nitpicking”
• “Splitting Hairs”
• “Seeing it from 30,000 feet”
• “Rising above it all”
• “You can’t see past your nose”

In other languages: Detail Biased

Japanese: “蟻の目” (ari no me) - “ant’s eye view”

German: “Gründlichkeit” - not just thoroughness, but a cultural value of doing things thoroughly and properly down to the smallest detail. Its base means “bottom”, or “foundation”.

French: “Les petits ruisseaux font les grandes rivières” - “Small streams make big rivers”

Russian: “Копаться как курица” (Kopat’sya kak kuritsa) - “To dig like a chicken”

In other languages: Abstraction Biased

Chinese: “不见泰山” (bù jiàn tài shān) - “Cannot see Mount Tai” - missing something obvious because you’re too focused on small things

Hindi: “जंगल में मोर नाचा किसने देखा” (Jangal mein mor nacha, kisne dekha) - “Who saw the peacock dance in the forest?” - suggesting that focusing too much on details can make you miss beautiful bigger truths

Arabic: “يرى الشعرة ولا يرى الجمل” (Yara al-sha’ra wa la yara al-jamal) - “Sees the hair but not the camel“‘

---

Conclusion

3D tools are a bit harder to get the hang of than normal HTML. I think that increased dimensionality of and interaction with information presentations is always more work. But I also think there are recyclable functions. To trace a spot on the surface of an object, to click and drag it, to pan and tilt an object - these are somewhat universal languages. The browser is very capable of executing them.

This display of information may never replace more mainstream, one-dimensional presentations, but I am interested in experimenting with it anyways in the hopes that it may be.

At the very least, I find this experience simpler. A bit stranger, in the case of my floating head, but with fewer hidden corners. I understand the extent of a space in a way I don’t always understand the extent of a webpage. It can hide so much in its links and pages.

This line of experimentation is still very unstructured, so I would love to hear your ideas, any of them! What would you like to explore when it comes to n-dimensional informational presentations and interfaces?