Of Bricks and Buildings

Bricks or Buildings
Discussing with my brother who is a Neuroscientist about how the discipline of science is responding to the influence of digital publishing and electronic networking between the practitioners.

Science production is slow piecemeal construction, brick by brick, of a wall of evidence which eventually becomes strong and tall enough to be a lasting structure. Each scientist toils away to make a few bricks – with experiments, replications, revised hypotheses, improved data. Sometimes (rarely) they get a chance to step back and synthesise all these bricks into their rightful place, allowing knowledge to move on a step, and providing a basis for the next layer of knowledge. The crucial process to ensure the strength of the bricks, and their ability to sustain the entire structure, is the rigorous examination of their quality by informed peer criticism. This takes place in face to face social rituals which are geared to challenging each scientist’s output and toughening it. Seminars and closed sessions in conferences. Named peer review in closed communications.

So far, online and electronic systems have not made inroads into these ritual processes. The forum of the physical seminar meeting – unrecorded and unminuted – allows for transactions in which criticism is advanced, tempered, withdrawn, adopted – without any earlier versions or procedural comments becoming part of the documented record. Scientists are trained in these rituals, know how far to push their critiques, and are not frightened of rigour or aggression because it is contained by agreed conventions.

Digital culture on the other hand fetishises the single utterance, by giving it status and permanency, connecting it to other significant discourse. Moreover, the online world lacks the machinery for stepping back from its proximity to specific digital objects and synthesising a wider and acceptable theory about a topic. When that happens, it’s usually at a social ritual (witness the recent web 2.0 Berlin conference consensus that web 2.0 has to grow out of flipping and instead create social value).

Also talked about the challenge of properly reviewing publications in science. For all bar the very top journals, reviewing is now effectively based on proxy criteria: what else has this person published, was it OK, does this paper have any obvious flaws. Adequate answers return a green light for publication. So this is a candidate for automation or automatons. The very top journals still operate a reviewing policy which demands that readers confirm there is original and significant science. That’s why citations in those journals are such coveted prizes.

The end of genuine editorial assessment ?

In my work on language testing, I am seeing the end of examination by an individual. Oral exams now test candidates ability to replicate a procedure.