My learnings from HalfStack's Conference
Fri, 22 May 2020
Some notes, key takeaways and hacky prototypes that I created during my first ever virutal conference! The conference was focused on UI-centric JavaScript and web development.
LEARNINGS FROM HALFSTACK CONF 2020
Building Motion Controlled Web Applications
Charlie Gerardā-āSenior (Front-End Engineer at Netlify) gave an awesome talk on how we can use some great new HCI devices like the "Myo Armband" to change the way we interact with our browsers.
Common HCI
- Keyboards
- Touch
- Swipe
- Voice
Incoming HCI
- Gyroscope
- Leap Motion
- Myo Armband
JS Camera Libraries
There are a crazy number of JS libraries that make use of a webcam. If you're working in ReactJS theres an awesome npm package called "react-webcam" thats great for getting started. We explored some libraries that can make use of webcam data in detail below.
Face Mesh
Uses key-points to get X,Y and Z axis. Uses a model to make predictions on the video source about the presence of the face.Ā Returns a list of different parts of the face, bounding box of the face and mesh data. (Lots of arrays!)Ā Built POC app called face-scroll that allows you to scroll up and down the page using your head
PoseNet
Pose detection. recognises body shapes. Pulls in video feed from clientā-ācan disable audio. Uses a detect function to guess pose based off of identified body shapes. Has a confidence thresholdĀ Made beatsaber clone in the browser.
Hand Pose
Hands only! Recognises key-points of the hand and has Z axis. Can recognise when fingers are folded. Syntax is similar to PoseNet.
Putting It Into Practise
I integrated pose recognition into my site's presentation clicker to create touch free slide navigation for presenters using just their gestures. This prototype is built using socket.io, posenet and one of my Gatsbyjs boilerplates which you can find here.
Limitations
All of this is superĀ new - Google released in last couple of months so we should see more improvements in coming years. Max frame rate is 40fps right now. Smooth animations are normally around 60fps.
Its Heavy - Library is around 12mb for handpose. This of course, affects site performance.
Accuracy - Laptop webcams and not depth cameras and may not be super accurate. Lighting is super important to improve accuracy.
Privacy - Constantly tracking the user while recording videoā-āconcerns over privacy need to be considered.
UX - This is a new way to interact with the web and as such would need to be learnedā-ābut people would be excited and would probably adopt quickly.
Key Takeaways
These tools are really useful to help improve the accessibility of theĀ web.
We could in the future have a "motion-mode" like "dark-mode" on our websites. This could be a toggle feature that when on allows the user to navigate a website with any pose, gesture, facial expression.
We could use it to improve our health andĀ fitness.
We could create websites that encourage us to exercise or help us improve out yoga positions.
We could use it beyond the browser
JS is not just for websites in the browser but also the internet of things. Imagine the possibilities?
"HCI is broad so start smallā-āfocus on one piece of the puzzle. Use one library or device and have fun!"
Responsive Design in 2020
Kilian Valkhof (Makes Polypane) gave a talk on the history of responsive design and whats coming up in the future with media queries.
Responsive Design
The way a website adapts to different screen-sizes by reflowing and repositioning content as the available space allows. Responsive design means:
- Mobile first
- Media queries width in ems
- Base font size
- Allow resizing
- Scrolling is fineā-āno fold
Cheap Dark Mode
@media (prefers-color-scheme: dark) {
:root {
background: #111;
filter: invert(1) hue-rotate(180deg);
}
img, video {
filter: invert(1) hue-rotate(180deg);
}
}
prefers-reduced-motion
Users can indicate they want to see less happening on the screen. Reasons they may want to do this may include motion sickness or maybe they don't want to wait for your animations to finish. This doesnt mean you can't use any motion, just be mindful.
prefers-reduced-transparency
Users can indicate they want to see things on solid colours. Reasons they may want to do this may include visual impairments but it can also help people who struggle to focus. This feature is not yet widely supported.
Network conditions: navigator.connection
We can use navigator.connection to determine whether a user would like to save their mobile data but also their effective network conncetion speed. You can see a simplified version of how this might work next to this!
Testing Network Conditions
if(navigator.connection.saveData
|| navigator.connection.effectiveType
=='slow-2g') {
preloadVide = false;
}Putting It Into Practise
The div next to this section has been set up using a media query to ensure that it always sticks to the users dark-mode preferance. You can use the dark-mode toggle at the top of the screen to switch it. It also detects network connection strength.
Your effective network type: Not supported by your browser!
Onto the next one!
Internationalisation and Data Visulisation
Naomi Meyer (SDE at Adobe) gave a cool talk on internationalisation and data visulisation in a COVID world!
"Data visulisations are powerful international communication tools to effectively communicate a storyand share critical information across the world."
Formatting a graph for the world:
Format integers correctly across each distinct local
Different locals can have different formats for displaying numbers correctly.
Think about colours
Different colours can have different cultural and regional signifigance so keep it in mind!
Dates and Times
Dates are often overlooked when formatting.
Don't assume left to right
There are many countries out there where people read right to left and its important to consider this when looking at these locales.
Culture
Culture is deeply rooted in our thinking patterns. Culture can effect how are uses interact and benefit from digital experiences. Internationalisation goes beyond language. By acknowledging cultural and social differences we're creating with innovation.
Touch screens were intially built for accessibility but now we are all reaping the benefits.
Reusable Formatter Instance
const reusableFormatterInstance = (locale, number) => {
return new Intl.NumberFormat(locale).format(number);
};
reusableFormatterInstance('en', 123456789123456789);
Integers
Integers are not the same across the world!
English --> 123,457 | Tamil --> 1,23,457
Further Reading
Putting It Into Practise
The div next to this section contains a demo of using internationalisation to format a string based on various locales. It uses compact notation to ensure it will fit the space.
Original Number: 12500100
English: 13M
German: 13M
Hindi: 13M
Russian: 13M
Korean: 13M
Tamil: 13M
...
...
...
...
