I aspire to be someone like Anil Dash, once I’ve paid my dues in tech and the arts. Anil is one of my favorite modern polymaths. I spent a fair amount of my life pursuing the arts in a past life and now I’m somewhere between tech and culture. He raises awareness about issues that overlap and often go beyond tech, and dedicate significant man-hours to writing.

Code Newbies first acquainted me with him and the conversation between Anil and Saron blew my mind. Maybe it’s because the blockchain industry has barely started to scratch the surface about ethics. Or maybe I’m tired of the industry with its technocrats and arguments for market-forces-as-a-silver-bullet. Their conversation centered around (tech) products manifesting the values of their creator(s) and how as “tech, broadly, is the wealthiest and most powerful industry that has ever existed… we have the highest responsibility.” Anil encouraged me to embrace and not fight the values my parents inculcated in me. He pushed me to think more about morals and ethics the deeper I get into tech.

I tuned in on his podcast just the other day. Racism has been a favorite area of interest for me thanks to spending over a decade immersed in bboying and mingling with people of color, not being comfortable in my own yellow “skin”, and not understanding why Korean communities are like crabs in a bucket.

The podcast dealt with racial bias in tech. Racial bias is funny. It could easily get baked in regardless of intentions if the creators do not take time to challenge their own biases (and most of us don’t).


Three things stood out:

First, technology can intensify design discrimination and systemic racism.


Stories of racist algorithms anger people and attempts at improvement receive acclaim until you hear the solution is peppered with deception and invasion of digital rights. A new digital imperialism is here and every nation-state is participating—even under the guise of diversity.

We laugh when the issue deals with camera filters. Once we venture into pedestrian detection for self-driving cars, risk assessment for criminals, or social credit scores, suddenly it’s no laughing matter. Living in a homogenous society that has its biggest company using physiognomy during its hiring process offers no comfort when these processes can easily be replaced by machines in the future. We slap tags on inexplicable black boxes in the name of science, assured they’re more objective when they’re not.

For all the good of science, history has repeatedly demonstrated its ethical shortcomings. Solutions to real-world issues are not context-free algorithmic problems. Choices made in the course of history gave birth to the world we live in. This world already has racism baked in, and we see its many devastating effects: high crime and unemployment rates in certain locales, family members with criminal records, etc. Without understanding context, magnifying social problems through technology and automation is just as easy as solving problems. Principles do not change: technology that addresses fundamental issues brings about lasting social change; tech fixes do not.

The people working on technology do not have to be explicitly biased or endorse racism—being oblivious is enough for systemic racism. Discrimination does not have a nametag identifying itself. The personal motive is not the issue, there are harsh limits and reality to implementation. Add to the fact, being unaware of the limits of these technologies without a healthy dose of skepticism could just as easily send us towards a techno-dystopian future rather than utopian. The price to pay for following the rules without questions and chasing after convenience has never been so high.


Second, defining our question and its scope is far more important than solving a hard problem.


Criminal justice software is like any other software. It receives data as input and sends output to assist in the decision-making process. However, it does not take a genius to figure out numbers do not paint the whole picture. There are already companies who sell software that automate different processes of rendering justice. Thorough reviews about this kind of software are already readily available and since the justice process is long, detailed, and complex, software on the market reflects that fact and varies widely.

The big picture is that the United States has a mass incarceration problem. Software solutions attempting to solve real-world problems start by recognizing the complexity of the task at hand, that they need holistic solutions, and that no one solution will do the job.

Visiting a website of a company like Arnold Ventures we find:

The statement “to maximize opportunity and minimize injustice” recognizes one single change takes work across multiple fronts. There are broader contexts to zoom out to. We view a tool like the Public Safety Assessment (PSA), which judges use that predicts a person’s likelihood to re-offend or return to court if released, as just one tiny piece of a giant puzzle. Lawrence Lessig in his book Code and Other Laws of Cyberspace presents four regulators: architecture/code, law, norms, and market. Criminal justice software taken out of context is an expensive random number generator, blind to justice, and ignores major forces outside law and code that can have an impact.

As mentioned before, systemic racism is alive and real. For example certain communities, because of the color of their skin, are arrested at a higher rate although all communities indulge in “illegal” activities at about the same rate. Those who end up being arrested and are in pre-incarceration are generally poorer than their non-incarcerated counterparts. The inability to post bail bonds keeps feeding this vicious cycle. Finally, let’s not forget the individual decision-makers and their existing biases reinforced over the years.

“Perfect is the enemy of good.” Notwithstanding, no tool or suite of tools can instantly eliminate systemic or individual bias(es). On the side of perfect, it would be foolish for any organization adopting software to overlook statistics and put faith in an app to solve social problems once and for all. However, on the side of good, people or companies developing software can see how software fits into the bigger picture and reiterate to provide better information. All solutions are imperfect and hidden biases will be prevalent in social data points no matter what. But through analyzing existing solutions, we start to redefine our questions and their scope.

But there is no reason why technology should be limited to computers. Properly understood, any new and better way of doing things is technology.

- Peter Thiel, Zero to One

Making the world better begins with identifying problems. We define problems first then look at “technology” solutions. Sometimes, ignorance gives birth to giving too much credit to one cutting-edge tech solution rather than focusing on the problem to formulate the best course of action. We are vulnerable to the appeal to novelty fallacy and nowhere is this more apparent than technology. Technology, to a certain extent, is driven by [hype cycles[(https://en.wikipedia.org/wiki/Hype_cycle). VR/AR, big data, and space travel have all had their hype season and are now in different phases but artificial intelligence, the poster child of modern tech, is probably the most misunderstood.

AI has clear limitations and in situations where one single asymmetric decision may lead to life-changing consequences like time in jail, AI does more harm than good. People can perform static analysis of software that does not use AI. The data you put in affects what you get out, you know how it happened, and another person can replicate the process granted she has access to the same data sources. There is no doubt AI is extremely powerful, and rapid technological progress has been made in areas like pattern recognition. However, in automating judgment and predicting social outcomes, reasonable people can disagree about the correct decision. Here, AI is downright inaccurate. No algorithm or massive amounts of data is ever going to correct systemic human biases or help predict the future.

Because tools hold authority and are seemingly infallible in this age, building accountable and transparent systems that better assist decision-makers take center stage. Never has objective transparent processes been more important than in the age of automation.

Judges do not make decisions, especially criminal trials, after referencing a single number spit out by one algorithm. The judge and her qualitative experience balances out the quantitative input from an algorithm and take into consideration all sorts of information before she reaches a decision. Modern technology acts as leverage to amplify the effects of decisions and the criminal justice system is an area we need to be wary of outsourcing decision-making.

Data obtained by observing problems come from local sources. Using those data sets, programmers write software that applies to the respective locales. Considering, no software magically works perfectly out-of-the-box, jurisdictions take validation steps to make sure data matches experience. Improving fidelity of software is a no-brainer and “that community understanding how that risk assessment works, how it’s predicting, and how it’s informing decisions” is a must.

When jurisdictions improve variables outside of software, they collect better data for software. Incremental progress then compounds towards solving actual problems with software as just one new variable in the process. The other variables include dated policies and practices of local jurisdictions. Perfect jurisdictions do not need software, and it is flawed jurisdictions with areas to improve that adopt software solutions. Progress can be made towards solving difficult problems even without using the latest artificial intelligence if questions and scopes are well defined.


Third, technology is neutral, but as long as it involves people, it sure as hell isn’t.


We are not going to witness technology disappear into the sunset anytime soon. Individuals, organizations, and nation-states infatuate over technology and people write non-stop about the utopian or dystopian future which await us. Tech is an invisible force that will usher in the future. We are to passively accept it with open arms. But one fact cannot be overlooked—we are in the driver’s seat for the future.

An all-knowing, all-powerful machine must originate from a programmer—with all her flaws. Likewise to how a child reflects and carries the mistakes and glories of the parents in her DNA, technical creations reflect human concerns which include biases, values, and assumptions. Technology, in theory, maybe neutral but in the real world, it is as spotty as the humans involved.

An individual and her casual decision-making during the day never left such a mark throughout history. A person’s circle of influence during her whole life was the extent of her village social circle. A wrong decision put her reputation in jeopardy but the effects were checked. We live in an unprecedented world where a single man can influence one-third of the world’s population and opinion. Add to this mix fallible, erroneous, and limited human beings and a world where wrongs do not look and feel wrong. The second and third-order consequences of a casual choice may lead to devastating effects.

Even with noble and pure intents, obliviousness to social reality and ignorance of historic patterns reproduce discrimination and prejudice. Indifference or just floating by is enough to result in replicating various social wrongs, at scale. History of science and technology shows again and again that progressive champions of previous ages justify and normalize injustices.

Our intentions matter, but not as much as we would like to think. We have made progress to the extent where wielding technology is the easiest it has ever been in history. On the flip side, our responsibilities to examine our own opinions have increased manifold.

The passage of time uncovered the limits of our scientific forefathers, and in due time, the limits of “modern” science will also be made evident. The antidote we have available for this condition is a tried and true one - a healthy dose of humility to never stop questioning science and technology.