← Home

Epic 2017 day 1 conference notes

I am attending the Ethnographic Practice in Industry Conference from October 22-25. Here are my notes from day one of the conference.

Fundamentals of Observational Research

How to alienate yourself sufficiently from the context so you can find what’s interesting?

What to observe? generally falls under 4 categories - People, Things, Actions and Settings.

There are a lot of interesting overlaps between reporting and observational research. Maybe the key difference is output and purpose?

Some techniques of Observational Research: Counting, Timing, Diagramming and Mapping.

Observational Research is about constructing your own data sets. But, the use of those data sets is very different from quantitative data analysis/data science. With OR, the purpose is that through the process of observing and creating the data set, you spark your own curiosity and generate questions and hypotheses.

Notation of your field notes: always include location, date, time and time span, and what’s being observed. This is necessary for turning your observations into data sets later on.

When including additional details: What’s the purpose of your notes? Answering that will let you figure out what’s the narrowest set of data you need (you want to avoid being overwhelmed/overloaded with too much information.)

Purpose of note taking is both to document data and also to help you focus.

(Q: How can you do observational research in digital contexts?)

With OR, we’re getting a feeling here that speaks in numbers. Don’t try for it to be statistically significant or representative.

Observational research is a way to figure out what your focus shoudl be through trial and error

Some tips and tricks: Start simple, have clear goals and hypotheses, list out things that you might want to observe and keep that list around as a guide, beware of the fire hose of information.

Ethnographic research design

Research process as design process - becoming designerly

It’s very hard to both be producing knowledge and producing things at the same time. (i.e. data analysis and storytelling)

Can think about design process as: Understand–>Create–>Execute

Researchers generate Thick Understanding–>Actionable Objects–>Precise measures

Can think of research along 2 axes - designerly <-> scientific, and qualitative <-> quantitative.

On creativity (one of the 4 Cs of design): Intellectual mood board of methods

Why it becomes hard to advocate when you don’t have precise measures:

When advocating and running into problems. Use left hand/right hand method. Left hand=what was literally said. Right hand=subtexts, both yours and the other person’s (Kassandra is the patron saint/goddess of researchers)

How to convince of rigor:

Keeping files: The key is reduction and synthesis. Also, consistent effort.

Key to producingn quality research designs: Must have credibility, analyzability, transparency and usefulness.

Doing Design Research in a Cognitive World

What does AI mean to you?

Dawn Nafus - Intel

AI: Discursive and engineering Discursive - where it’s at is v early stage and eveyrone has to have an opinion

See two directions:

  1. An opportunity for us to do slow data. Really spend time with the input data that goes into the machine learning algorithms

  2. Sharpen our instincts with respect to the kinds of jobs we say yes to vs the jobs we say no to. Who do we enable to control the data and the narrative

Christian Madsbjerg - ReD Associates

There’s almost a death wish in the technology world: “If only we could build a machine that could generate truth, then we can stop thinking.”

With the computers we have now, we could feed it 40m pictures of dog and it would know what a dog looks like. but i can do that with one picture with my daughter. I’m more interested in people.

We’re under-indexing on understanding people and over-indexing on understanding machines.

I just think we should spend a bit more time observing the world rather than looking at a screen.

Mark Burrell—Watson Health, IBM

70s and 80s, lots of discussion about not just artificial intelligence but more generally about what intelligence is.

Think there isn’t just one type of intelligence.

One definition of intelligence - an ability to achieve complex goals or problems.

But issue is different types of problems - some more clearly defined, others less so, more ‘wicked’

If we’re trying to help physician and patient decide best treatment for a type of cancer, what if no research exists? If tried best practices and doesn’t work? Physician and patient can also disagree on what ‘best’ is - this is an example of a wicked problem.

Don’t like to talk about ‘artificial intelligence’ because it’s not about replicating and replacing human intelligence, but how to complement address limitations of humans - i.e. no way I could crunch through all the data, or read all the research papers. waht if computer could help by just telling me what’s new?

Where are we with regards to the promise of AI?

Dawn: Need to be aware of whether AI or AI-driven systems are designed inclusively, with people who are affected able to provide feedback and be counted.

Mark: Often need people to provide the ground truth. On some systems that are more important, need decision support system: transparent and explainable.

Are we building appropriate trust - neither too little or too much so they imbue it with magical qualities

Melissa Cefkin - Nissan Research Centre: Autonomous systems are relational systems. What’s changed is the structures and systems of our work, and how we are equipped today to do the work to craete such systems

With the distributed nature of the internet it’s possible to annotate and create more input than before.

Christian: Two jobs - one is extension of man. but isn’t there a job for AI itself to make sure it doesn’t tank again by overpromising massively.

Dawn: Work practices. One area I do have some hope is, having some data in hand that we can all look at could help. Won’t shake strong ideology but could be a better approach.

Melissa: What does it mean to do more ethnographic type of work when it’s so foundational - 8 years lead time between research and product on the road.

Autonomous systems are essentially relational systems. We need to be involved in the shifting of relationships

newer/newish useful work - Can we put in place measurement and evaluations systems that can play a role in affecting the output.

Christian: There’s a place to be critical of the ideology, for example with agile.

Dawn: one reason why temperature controls get made into many products is because it’s relatively easy data to work with (and therefore we’re going to scope our problem accordingly). But instead should ask whether this is a problem we should be working on at all.

Lots of unexpected allies - data scientists, for example. There is a course at UC Davis on Data Archaeology taught by both an anthropologist and data scientist - why is that column there?

Christian: If you go back 5 to 10 years ago this conference was all about design - how to work with and influence designers. Today, all we’ve talked about is how to work with engineers.

Dawn: On ethics of AI - one important thing is to make it very concrete and very clear when an ethical decision is being made.

Mark: One important role of research: How to help the people you work with figure out what problems to solve.

Q: how do we use ML/AI to do better ethnography?

Dawn: Don’t really know, but one example of when she wished there was computer vision/AI was when sorting through lots of images. in general - useful for automating data analysis/processing but don’t let it make decisions for you.

Melissa: there’s no general AI at the moment they are all very targetted, so you have to be very clear about what problem you are trying to solve.

Christian: introduce different people to the process. reality is a powerful thing if you introduce it.