Getting Out of Our Lanes: Understanding Discrimination in the Digital Economy

Lambert here: This is important work. I share the perspective that “technology is creating more warped forms of discrimination.” However, by focusing only on surveillance, the author misses a real opportunity to connect this perspective to the foreclosure crisis, where elite accounting control fraud targeted and disproportionately affected minorities, and where “technology” in the form of the bent database design of MERS and criminal robosigning of scanned documents played a key enabling function. It’s not an accident that something like 50% of the homes in Ferguson are underwater, and that the City of Ferguson’s happy policy of raising revenue by arresting and fining its citizens, among them members of home-owing families, causes great pain, distress, and anger, as money that was put in the kitchen table coffee can for the mortgage goes to fines and bail.

By Raúl Carrillo of iNet, a J.D. student at Columbia Law School. Originally published at New Economic Perspectives.

In the fall of 2013, on the 50th anniversary of the March on Washington for Jobs and Freedom, Ohio State University Law Professor Michelle Alexander penned a brilliant essay in The Nation, entitled “Breaking My Silence¨. In the piece, Alexander, author of the groundbreaking book, The New Jim Crow, urged social justice advocates to get out of our “lanes” and “do what Dr. King demanded we should: connect the dots between poverty, racism, militarism and materialism.”

In this spirit, I am writing to encourage readers to take up yet another task, one I’ve unfortunately only recently shouldered myself: to understand how digital surveillance reinforces socioeconomic hierarchies.

As writers on this site argue every day, despite its strikingly clear ability to do so, the U.S. government does not fund the public infrastructure necessary for sustaining basic human dignity. Thus, most people striving for economic security in this country can only look to private institutional creditors. Today, this usually means also interacting with Big Data. Yet for certain groups of people—the usual suspects—social surveillance not only jeopardizes civil rights, but threatens financial predation and exclusion beyond the norm. In the age of the internet, injustice is increasingly intersectional.

In 2015, even the mainstream media harps on invasions of privacy. At this point most Americans are aware of the existence of data-mining, if not its full extent. This is partially because the practice touches most everyone. Even some of the most privileged people’s information is siphoned so companies can barrage us with advertisements, assaulting our anonymity and autonomy, while supposedly compensating us with consumer convenience.

Yet this harvesting of personal data is not merely irritating or unsettling. It outright punishes many people.

Protected Prejudice

Last spring, in a piece entitled “Redlining for the 21st Century”, Bill Davidow of The Atlantic sketched some ways in which algorithms deny people financial services, or charge them much higher rates, based on their inferred race. More specifically, watchdogs have noted that mortgage providers can now determine an applicant’s ZIP Code via their internet access, and then proceed to charge them based on the neighborhood where they reside. If human employees did this, they would be acting in clear violation of The Fair Housing Act of 1968. Yet, because algorithms are the actors, the practice often goes unexamined, and people who might otherwise receive credit at a reasonable rate get hurt.

Humans write the code. Humans set up the servers. Humans make sure everything hums. Yet despite these obvious facts, there is a lengthy legal history of presuming machines to be independent and untainted. For example, University of Maryland Law Professors Danielle Keats Citron and Frank Pasquale have recently detailed how credit scoring systems, despite their supposedly objective simplicity, are “inevitably subjective and value-laden.” Although the systems were initially built to eliminate discriminatory practices, many are now accomplishing the opposite function. Systems can only be as free from bias as their software, and thus only as righteous as the values of developers and programmers. Even intent hardly matters, though. As mathematician Cathy O’Neil has written, seemingly neutral choices can have a disparate impact.

Digital discrimination isn’t limited to the private sector, either. In an essay entitled “Big Data and Human Rights”, Virginia Eubanks, an Associate Professor of Women’s, Gender, and Sexuality Studies at SUNY Albany, notes that although New York State social service providers initially used algorithms to expose the discrimination of their own employees, the tables have turned. Faced with the fiscal burden of supplying benefits in the wake of recession, the state commissioned technologies that replaced the decisions of social workers, allowing bureaucracies to deny benefits behind the illusion of angelic machines. Now, computers, rather than human employees directly, can make choices about social spending based on built-in prejudices regarding welfare recipients.

The law generally ignores all of this. In his profound new book, The Black Box Society, Pasquale writes that although the majority of people hurt by the structure of a particular form of surveillance may be Black, Latino, women, LGBTQ, or people with disabilities, the tools used are almost entirely immune from scrutiny. Title VII of the Civil Rights Act of 1964 has been deemed “largely ill equipped” to address this sort of discrimination. There are few applicable laws to enforce–or at least laws that protect our privacy rather than corporate ways and means. For example, in the case of credit scoring, many credit bureaus are not required to reveal how they convert data into scores. In exquisitely Orwellian fashion, those processes are deemed “trade secrets”, shielded from the eyeballs of debtors.

Contaminated Commons

In my last post for New Economic Perspectives, I argued that if one views the financial system through the lenses of Legal Realism and Modern Money, financial firms serve many of the functions typically associated with public utilities. One arrives at the conclusion that, in a sense, when chartered lending institutions create credit for certain people and not others, they are privatizing pieces of a commons. This fact is not bad in and of itself, but it should make us think critically about the financial-legal-digital matrix now increasingly mediating our lives.

For example, because there is no “equivalent of Medicare for housing” in this country, as Dr. King called for, many people rely on government-supported private lenders to finance their housing needs. In fact, politicians have increasingly voted to subsidize the private mortgage industry rather than create a true public option for housing. If, because of data-mining, some people have reasonable access to mortgages and others do not–say, merely because they are Black, or pregnant, or gender nonconforming–this process becomes tantamount to government-sanctioned denial of access to housing. Many people will rent space from a landlord if they can’t get a mortgage, of course, but the point is that broad swathes of the population are pushed toward a private market, and then subsequently shut out of it.

And so it goes. If social justice advocates don’t understand digital surveillance, we will always be one step behind at best. Government spending and private lending are inherently discriminatory. To a certain extent, they always will be. Yet technology is creating more warped forms of discrimination–in addition to reviving old ones. As Modern Money friends Sandy Darity and Darrick Hamilton have often noted, a large part of the racial wealth gap can be explained by redlining, as well as general housing and lending discrimination. These practices were supposed to have been eradicated by the Civil Rights Movement and subsequent legislation, at least allowing people to be discriminated against merely because they were poor, and not because of other factors. In the absence of massive social spending, we’re all supposed to get an equal opportunity shot at credit. That’s part of the American Dream. But the advances of minorities may be barred simply by operating through the internet.

Data-gathering technology is rendering assaults on civil and economic rights increasingly difficult to untwist. Therefore, privacy advocates should provide auxiliary support to other social justice movements, especially #BlackLivesMatter. Equality advocates should take up the call to check government and corporate surveillance.

There is plenty of anger in this country at invasions of privacy. There is justified outrage at inequity and inequality. Now there must be synthesis and alliance. As Michelle Alexander argued, we have to get out of our lanes. Indeed, we may no longer have a choice. When I was younger, it was trite to say the internet was an information superhighway. Today, the digital economy is an information labyrinth. We have to navigate it together: the idea of isolated lanes doesn’t even make sense anymore.

Print Friendly, PDF & Email
This entry was posted in Guest Post, Income disparity, Politics, Surveillance state, The destruction of the middle class on by .

About Lambert Strether

Readers, I have had a correspondent characterize my views as realistic cynical. Let me briefly explain them. I believe in universal programs that provide concrete material benefits, especially to the working class. Medicare for All is the prime example, but tuition-free college and a Post Office Bank also fall under this heading. So do a Jobs Guarantee and a Debt Jubilee. Clearly, neither liberal Democrats nor conservative Republicans can deliver on such programs, because the two are different flavors of neoliberalism (“Because markets”). I don’t much care about the “ism” that delivers the benefits, although whichever one does have to put common humanity first, as opposed to markets. Could be a second FDR saving capitalism, democratic socialism leashing and collaring it, or communism razing it. I don’t much care, as long as the benefits are delivered. To me, the key issue — and this is why Medicare for All is always first with me — is the tens of thousands of excess “deaths from despair,” as described by the Case-Deaton study, and other recent studies. That enormous body count makes Medicare for All, at the very least, a moral and strategic imperative. And that level of suffering and organic damage makes the concerns of identity politics — even the worthy fight to help the refugees Bush, Obama, and Clinton’s wars created — bright shiny objects by comparison. Hence my frustration with the news flow — currently in my view the swirling intersection of two, separate Shock Doctrine campaigns, one by the Administration, and the other by out-of-power liberals and their allies in the State and in the press — a news flow that constantly forces me to focus on matters that I regard as of secondary importance to the excess deaths. What kind of political economy is it that halts or even reverses the increases in life expectancy that civilized societies have achieved? I am also very hopeful that the continuing destruction of both party establishments will open the space for voices supporting programs similar to those I have listed; let’s call such voices “the left.” Volatility creates opportunity, especially if the Democrat establishment, which puts markets first and opposes all such programs, isn’t allowed to get back into the saddle. Eyes on the prize! I love the tactical level, and secretly love even the horse race, since I’ve been blogging about it daily for fourteen years, but everything I write has this perspective at the back of it.


  1. Kevin Smith

    Excellent, enlightening and disturbing analysis.
    Another good reason why I start my day with NC.

    Best wishes,


  2. Paul Harvey Oswald

    At the risk of stretching the “lane” metaphor too far, see the EFF report on Automatic License Plate Readers in the links section for more evidence.

  3. McMike

    I think the ultimate example of big-data enabled discrimination is high-tech modern gerrymandering.

    Meanwhile, the author raises an interesting point about privatization. Indeed our government would rather subsidize than socialize. Setting up an array of subsidies for mortgage lenders, landlords, and rent-seeking middlemen of all stripes (think debit cards for benefits and e-filing for taxes). Investing massive amounts of public treasure into corporate subsidies, guarantees, preferences and forbearance – putting the massive will and might of the government behind these private players – always and extensively to the detriment of public alternatives, and invariably without much strings attached in terms of public safeguards.

    The government is perfectly willing to put massive amounts of taxpayer money and might to work… as long as it is done to channel citizens into corporate extraction lanes; like sheep to the slaughterhouse.

  4. Eureka Springs

    The internet is worse than a rentiers paradise beginning with the telco or cable or sat. providers, an intentionally narrowing and slow toll road, roving eye of Sauron or a mine field…it’s an assault weapon (read yesterdays must read and this). Hell I will have to pack up my computer and go rent a hotel for fast wifi if i ever want to upgrade my operating system. 8 gigs at home is impossible.

    As for credit for the poor… well that’s a double edged sword. Continuously expecting government to sort that out through privateers with better more complex code only adds edges – death by a thousand cuts. All neoliberal code expects humans to run/act like a machine without fail. When failure happens it’s never neoliberalisms fault. And if/when you fail you are nothing more than a carcass for the turkey buzzards. Ferguson is such an excellent metaphor.

    The system is rotten far past the head of the fish…. suggesting a few small tweaks as a solution here and there is a fools errand.

  5. MartyH

    There is much written about this and much wisdom, including this piece. The 2¢ I would add relate to:

    As writers on this site argue every day, despite its strikingly clear ability to do so, the U.S. government does not fund the public infrastructure necessary for sustaining basic human dignity.

    Good luck spurring public-sector energy for that initiative. They’re busy killing off the last of the New Deal. The wire to the house cartel was handed “The Internet” as an opportunity to offset the erosion of the value of their physical plants. Wireless was obviously going to destroy copper and fiber infrastructure’s economic strangle-hold unless some new monopolized demand could be created. Enter the Internet and Cable. This is a tiny edge-case in the privatization of the infrastructure necessary for the sustaining basic human dignity. Tiny, but valid.

    In links today, Tim Wo’s “Small is Beautiful” and Colin Dickey’s “A Flaw In Our Design” provided counterpoints to this thought for me. Competition among initiatives striving for useful differentiation along identifiable competing value choices strikes me as a good thing.

      1. MartyH

        Error was that Small is Beautiful in today’s links was by Tim Wu, candidate for Lt. Governor of NY in last November’s election. Yes, I have a copy of Schumacher’s book of the same title too, diptherio :-P

  6. lightningclap

    I would love to see someone with expertise examine and expand on the concept of “Code Is Law”. MERS is the most obvious example, but many aspects of our lives are affected by supposedly neutral algos. Discrimination, theft, fraud, etc. can be written in (feature not a bug) and there’s no way to know (the code is proprietary), and nobody to answer for it (the ultimate control fraud).

    Thanks for tying it in to the repercussions in the real world. This is a largely unaddressed social issue.

  7. Nonanonymous

    The only thing that matters about digital surveillance or censorship is that we not allow it to happen. Not now, not ever.

  8. c1ue

    As someone who is working with LPR companies – the reality is far different than perceived capabilities of LPR.
    Yes, there are LPR databases of 700 million or more license plates already in existence. However, there are only 250 million registered cars. The Big Data storage technology and Big Data analysis algorithms make retrieval of this much easier than before, but this is still a far, far cry from anything that anyone could describe as comprehensive. I will note that I would specifically exclude national intelligence capabilities from this description.
    There are, however, many legitimate uses for LPR in terms of building data for good. Parking in many cities is a gray area almost exclusively built upon an edifice of historical practice, back room politicking, and reactive policy to “squeaky wheels”. Creating the data structures and using LPR to populate behavioral information would allow much greater transparency and actionable intelligence to review existing policies.
    For my part, I personally push very hard for strong anonymization so that the benefits can be severed from the invasion of privacy – but I can tell you that few companies normally bother.

  9. washunate

    I really enjoyed this article, but it still stays remarkably within the lanes. Zip code based mortgage discrimination is a relatively minor symptom. The issue is that we have such a stratified class system today that where you grow up determines everything about you, from your likelihood of having computer access at all to whether it is more likely you will graduate from college or get arrested.

    There is no way to address discrimination in housing without ending the drug war and restoring rule of law.

Comments are closed.