rule society and our lives (R0ssaak, 2016, p. 34). Compared to human processing,
computational algorithms have many advantages since they are much faster, can
deal with more complexity and are more accurate than humans will ever be. The
downside of this computational processing is that these systems rely on processes
and abilities 'that are radically beyond what is possible for human beings to
understand' (Danaher, 2016, p. 247). They are black boxes and that is what gives rise
to many concerns, because we do not understand how these algorithms operate as
the new power brokers in society (Diakopoulos, 2014, p. 2). Critics like Evgeny
Morozov and Cathy O'Neil stress that algorithms are constructed models, based on
choices what to include and what to leave out. And these choices 'are not just about
logistics, profits and efficiency. They are fundamental moral' (O'Neil, 2016, p. 218;
Morozov, 2014, p. 182-186). John Danaher warns that the increasingly reliance on
algorithms in decision making processes might turn society in an 'algocracy', a
governance system in which computer-programmed algorithms are used 'to collect,
collate and organise the data upon which decisions are typically made, and to assist
in how data is processed and communicated through the relevant governance
system' (Danaher, 2016, p. 247). While in a bureaucracy laws and regulations
structure and enforce how humans act, in an 'algocracy' the algorithms are the
structuring and constraining components. Janssen and Kuk emphasise that
algorithms do not work on their own, but form an 'algorithmic materiality', which
means that there is an intricate relationality between algorithm, systems, data and
humans resulting in a dynamic and 'complex socio-technical ensemble of people,
technologies, code developers and designers' (Janssen Kuk, 2016, p. 274-275),
which is very similar to the archive as 'the apparatus through which we map the
everyday' (Giannachi, 2016, p. xv).
A few societal examples discussed
There are good reasons to worry about this emerging 'algorithmic governmentality'
as some scholars label this data-driven exercise of power and policymaking (Thomas
Berns, 2013; Rodrigues, 2016). Before discussing the archival implications of
these socio-technical developments, I want to review some examples. Real time
processing of large quantities of data from criminal records, police databases and
surveillance data to predict where criminal activities are likely to happen (predictive
policing) has already been put into practice in several countries (Joh, 2015). Even in
the courtroom computational algorithmic support has been introduced to underpin
court decisions. The independent non-profit organisation of investigative
journalism ProPublica, recently published a series of critical articles on the
accurateness of algorithms used in courtrooms to assess the likelihood of recidivism
of defendants. ProPublica journalists analysed the accuracy of a widely-used risk
assessment tool named COMPAS (Correctional Offender Management Profiling for
Alternative Sanctions) by investigating 10,000 criminal defendants in Florida and
compared their predicted recidivism rates with the actual rates. The researchers
found out 'that black defendants were far more likely than white defendants to be
incorrectly judged to be at a higher risk of recidivism, while white defendants were
more likely than black defendants to be incorrectly flagged as low risk'.4 Although
the Supreme Court of Wisconsin expressed its concern about this race correlation in
COMPAS, in an appeal from an order of a circuit court it judged that the evidence-
based risk assessment tool COMPAS can be used at sentencing.5 In the explanation
of its decision, the Supreme Court circumscribed its use by stressing that risk scores
are 'not intended to determine the severity of the sentence or whether an offender is
incarcerated' and that risk scores 'may not be considered as the determinative factor
in deciding whether the offender can be supervised safely and effectively in the
community'. Although the Supreme Court agreed that the defendant-appellant was
not able to review and challenge how the COMPAS algorithm - which is part of the
trade secret of the developer Northpointe Inc. - calculates risk, the order judged the
ability to review and challenge the resulting risk scores as satisfactory.6
Interestingly, one of the judges, Shirley S. Abrahamson wrote a separate
consideration in which she emphasised the relevance of recording the use of risk
assessment tools. Precisely because scholars were critical on using these risk
assessment tools in sentencing, courts should 'evaluate on the records the strengths,
weaknesses, and relevance to the individualized sentence being rendered of the
evidence based tool (or, more precisely, the research-based or data-based tool)'.
Abrahamson recognised that this might be an extra demand on and administrative
burden for the circuit courts, 'but making a record, including a record explaining
consideration of the evidence based tools and the limitations and strengths thereof,
is part of the long-standing basic requirement that a circuit court explain its exercise
of discretion at sentencing'.7
We need to question how the record is defined if the judges accept that algorithms
can be used in sentencing although the algorithm itself, the lens through which the
data are filtered, sorted etc., remains a black box because of the mentioned trade
secret. The record-making as defined by the Supreme Court has to do with
accountability of how the judges use the tools in the process of sentencing, not with
the processing activity of the algorithms themselves. This appeal clearly shows the
limitations of the traditional scope of the concept of the record. If the informational
algorithm remains a closed black-box in cases with far-reaching consequences for
citizens (even if the outcomes can only be used as additional information for
decisions) the claim that records provide the best means for warranting
accountability is severely affected. The ever-increasing interrelationship between
man and technology requires a clearer notion of the scope of the record, especially
when a relation is made to accountability of decision-making. There are good
reasons to redefine the scope of the record in that tight relationship between
humans and machines. I agree with Amelia Acker, who argues that examining the
infrastructure of records, 'archivists can think big enough about the "black box" and
all the layers of construction behind digital records and emerging documentation
practices' (Acker, 2016, p. 294-295). One of these layers of construction are the
algorithms that are used in the processing of data. The use of 'black-box' algorithms
in decision-making processes will be mirrored in the records that are created, and it
is not without consequence to the attributed quality of the records as means of
accountability. It is imaginable that for some decision-making processes (which
immediately shows an additional selection perspective) open and understandable
algorithms are required. That is for instance the motive of a motion for the European
archives in liquid times
202
charles jeurgens threats of the data-flood. an accountability perspective in
the era of ubiquitous computing.
4 https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm accessed at 30
March 2017
5 Supreme Court of Wisconsin, State of Wisconsin versus Eric L. Loomis on certification from the court of
appeals, 13 July 2016, available at <https://www.wicourts.gov/sc/opinion/DisplayDocument.
pdf?content=pdf&seqNo=171690> accessed at 30 March 2017.
6 Ibid., par. 53.
7 Ibid., par. 141
203
