Just before Xmas in 2017, having had a minor operation, I was lying in bed, bored, uncomfortable and in no mood to read, when for reasons I can’t explain, I thought I’d listen to Keith Jarrett’s The Köln Concert all through. I’d played it massively over the years, on record, cassette, CD and now on download. But since I tend to listen to music while doing something else, reading, writing or looking out of the window, I can’t say I’d actually heard or immersed myself in it for a long time.

I recalled that the double album was recorded live in Cologne in 1975 and a lot of people bought it, even those who would never have listened to anything quick or abstract by Charlie Parker or John Coltrane. Musicians – not novelists, movie or theatre directors or painters – had been at the centre of the culture I grew up in. They were our political and spiritual guides and we considered it crucial to keep in touch with what they were thinking. Despite this, for some unknown reason, I didn’t hear the album until around 1987.

I remember being depressed at the time because I told everyone so, and then I left London briefly, to stay with Karen, in Cardiff, Wales. In the early seventies we had been at college in Bromley together, doing our A levels, and she was the first female friend I’d had. It was not a romantic attachment: better educated and more cultured than me, she was right of centre then, argumentative and good fun to be around. She came often to the house and my father liked to talk with her. He encouraged us to start a magazine together, which had one issue.

In Cardiff, expecting her to welcome a definitive account of my depression, I arrived to discover that she had married a Buddhist and become one herself. They were wearing orange robes, burning incense and sitting still for long periods. I was appalled: how happy they were despite being ridiculous, having no drugs in the house and wearing a colour that did no one any favours. I grasped that not only would I receive little attention, but that afternoon they also wanted to see Raiders of the Lost Ark, which had just been released. Of course, its optimism hurled me into a blacker mood. A good Tarkovsky might have elevated my spirits.

My mood would turn darker that evening. Buddhists, according to my bias, tended to listen to chanting music or something somnolent suitable for aromatherapy. The new Thatcherite capitalism was depleting and exhausting people. If you couldn’t keep up, you could cross your legs and check out. Nietzsche called Buddhism ‘a kind of hygiene’. But I was half-asleep already; I wanted to wake up and tune in to the cosmic meaning. That was why I was there.

Karen’s husband, whom I had taken against – particularly after the devoted way I saw him hold and caress her foot when we visited a shoe shop – put on The Köln Concert, which I knew nothing about. When he informed me that the piece was an improvisation I suspected my visit would be short. In the seventies and early eighties I had worked in the theatre and at that time directors liked to use improvisation to ‘free the actors up’. These exercises were interminable; I had never seen an actor achieve anything through improvisation that wouldn’t have been better coming from a writer who had thought about what he was doing. But, having no choice – ‘the ears have no lids’, as Jacques Lacan reminds us – I listened. And those first five notes knocked me out; it was like receiving five firm blows to the head.

That night we had supper and a young woman friend of theirs came round. When the woman had gone and the Buddhists were in bed I retired to the attic where I was staying, and waited. In those days I was an enthusiast of one-night stands, where a never-repeated intensity and strangeness might occur with an unknown person who would remain mostly unknown – apart from their obscenity. Hoping for what Robert Stoller calls ‘reciprocated pathologies’, the couple would become and remain a kind of living fantasy for one another. That was the idea.

Earlier, the girl and I had whispered together. Now she came back. We lit candles; I crept downstairs for the record. We lay on the bed together and played it all night.

I curled up. Everything was wrong with me. Suffering from lack of curiosity, I was too fearful and inhibited to have sex with her, if that was what she wanted. Or, indeed, if it was what I wanted. Sex rarely lacks trauma: it is almost always shattering and there are few insignificant sexual encounters, however fleeting.

But I lacked the sense or ability to inform her of how I felt. If speaking is obviously the most important thing we do, I could have tried that. She might, after all, have wanted to hear and respond.

Certainly I had always considered it more profitable and interesting to talk with men than with women. My father and I had been close, and I had witnessed how much he and his brothers liked to talk. In contrast, my mother avoided social situations and conversation. She was already nervous, if not frightened and trapped. Later I understood that the freedom to speak, joke or tease could never be enjoyable for her. Petrified, secluded and busy trying not to go mad, more talk would only disorientate her. She was enigmatic to us, and appeared to have little idea of what was going on inside herself. Not that she wanted to be helped. She didn’t think it was necessary that people be interesting, funny or attractive. On the rare occasions when people came by to see us I wanted them to have a nice time and like us. Once, when I was enumerating the qualities of someone I liked, she interrupted to say, ‘Why can’t people just be nice?’ My analyst said that that was a profound remark. Clearly, he was on to something. He’s nice himself, but I can’t imagine that people pay him for only that.

Yet I had always been fascinated with women’s bodies, their gestures, clothes, voices and who they were. But, that night, what I called depression was rigidity and repetition; a taste of bitter nothingness. I was lost and afraid in a dark wood with no capacity to enjoy my own thoughts or those of anyone else. I could enter a tunnel of all-debasing, tantrumy fury where things would get dirty in my head, and I’d be tempted to throw myself under a train. Who doesn’t know someone who has killed themselves, and even admired their courage? In these moods you can forget that you are the engine of your own tempest.

The nearly dead certainly lack a sense of humour. The British child psychoanalyst Donald Winnicott calls depression ‘smoke over the battlefield’, but where exactly was the conflict taking place? Who was the speechlessness – the block, the dire shortage of words – for? How do you begin to understand what is going on inside you? I’d travelled a bit and recognised that we lived – if you were in London, if you’d benefited from the welfare state and were in work – in a relatively free society where the bright new individualism of the sixties was still being celebrated, albeit in a darker form. With me, the sources of oppression were within. I was exerting a reign of terror over myself, while destroying my ability to resist. I had put myself on my back.

Stephen Frears and I had made My Beautiful Laundrette and Sammy and Rosie Get Laid. I had some money for the first time and not long before had been at the Oscars, sitting next to Bette Davis, who had been kind. Now I was at a crossroads, with no direction home or ahead. I knew I should begin the novel I had been attempting since my teens. It would become The Buddha of Suburbia, but I didn’t know how to start. I couldn’t find the right voice for it. Or a voice for myself.

Having successfully sabotaged the Buddhist couple’s weekend and more or less lost Karen as a friend, back in London I bought The Köln Concert on cassette because I needed to know it better. The sound is thin and you can hear Jarrett sighing, foot-stomping and making Glenn Gould-like grunts. He was twenty-nine and exhausted that night. The concert took place late, after another concert had finished, and Jarrett almost refused to play because the young promoter had provided the wrong piano. It was only by chance, apparently, that he played at all, or that the gig was recorded.

The Köln Concert is a textured piece: you can hear in it pop, gospel, blues and a little bit of schmaltz, but Jarrett never stays anywhere long enough to settle. He is all over the place. Not only did I understand that the music was original and somehow visionary, but that it was so rich – carrying in it the compressed history of everything Jarrett ever knew – that even today I can detect hidden corners in it. And who couldn’t acknowledge that it took a wild confidence to sit in front of a thousand people and invent music that didn’t exist five minutes before? Rapturous, possessed by music, he had – and yet hadn’t – exposed himself to an exhilarating danger. How could anyone be that free? Nietzsche, himself a keen improviser on the piano, has something to say about this in The Gay Science:

One is reminded of those masters of musical improvisation whose hands the listener would also like to credit with divine infallibility although here and there they make a mistake as every mortal does. But they are practised and inventive and ready at any moment to incorporate into their thematic order the most accidental tone to which the flick of a finger or a mood has driven them, breathing a beautiful meaning and soul into an accident.

It also struck me, as I drank in the record – whose beauty slowly began to convince me I needn’t kill myself that afternoon – that satisfaction and happiness wouldn’t happen for me today, or for a long time. My response to the music had reminded me that concealed inside myself was a more excitable and open self raring to get out. But I knew it could take years to chase away one’s defences and live more freely. You couldn’t take a pill for it. You had to do something. I had read in the Freudian literature that you had to ‘work it through’. How did you begin to do that?

My Beautiful Laundrette had been about two aspirational, gay oddballs – a skinhead and a mixed-race kid – who become entrepreneurs almost by mistake. It matched its time: extreme neo-liberal capitalism was gathering speed; society was being sorted into winners and losers. Where there had been once, at least, generational solidarity and a sense of shared, countercultural values – for the liberalisation of the West – there was now, in this new era, the cry of ‘no future’ and the banal mantra of celebrity and accumulation. Not only that, this was the era of self-help. You were supposed to retool yourself for the new era. In the light of this, even I grasped what was stimulating the Buddhists. If all was acquisitiveness, competition and entrepreneurship, sitting on your ass could seem like rebellion.

I was too restless and ambitious for extended meditation, and I regret that I probably still am. But I began to think about something related to meditation. Suppose you put aside, or entirely gave up, ideas of success and failure, and only proceeded experimentally, following your interest and excitement? What if you retired what Rousseau calls ‘the frenzy to achieve distinction’? Didn’t that resemble what Jarrett was doing when he turned up not knowing what would turn up? Wasn’t that a lateral, Buddhist act?

When I was at school we were neglected when we were not being punished. But now there were fresh horrors for children, and new forms of discipline. Every day had become a test. Goals were set. The young were harried and made to pursue some spurious ideal of excellence and achievement. Putting all this together, and recalling that refusal makes us human, I had been thinking of writing a story about someone who one day rejected the idea of duty and obligation, and decided to proceed only according to his pleasure, following what you might call ‘alternative selves’, seeking an ‘alternative life’, re-evaluating his values. He would, of course, soon exhaust the obvious indulgences. If he didn’t insist on making himself crazy like Dorian Gray, how might he proceed? Where would it take him, this commitment to surprise, and what would happen as a consequence?

I never found a way to order this story. But I wanted to try and write a book about my own discordance. Despite my state of mind, most of the time my discipline hadn’t deserted me. It would always come and go, but most days I would drift across to my desk, read a bit of what I had written previously and cross it out. Usually something kicked off then. Beginning a new piece created some hope and optimism. I embarked on the novel because I had to, starting it simply with the most elementary statement, having the protagonist announce himself, as we did in workshops. ‘My name is Karim Amir and I am an Englishman born and bred, almost . . .’

A few days ago, after listening to The Köln Concert, I was prompted to glance again at Keith Johnstone’s teaching book Impro, which we’d used at the Royal Court, where Johnstone had worked. Despite my scepticism with regard to exposing others to your improvisations unless you were Keith Jarrett, this paean to fluency and spontaneity made me so enthusiastic I read it again. I have to admit I adore books that contain instructions.

Freud was always a moralist. Trying too hard to be obedient or good, we become masochistic, because morality in its purest, Kantian form is pathological and asks too much of us. Winnicott, in Playing and Reality (1971), discusses the idea of the child being asked to give up their spontaneity in favour of compliance, and what the price of such obedience is. Published in 1979, Johnstone’s Impro, with its hippy edge, is an immaculate guide to not doing the right thing, to being careless, indecent – and altogether crazier. Wisely, Johnstone calls education ‘an anti-trance activity’, and suggests that by forgetting your manners and being less impressed by the rules, you can allow surprising things to happen. True speech might even emerge if we remember that speech isn’t something you can rehearse; it really is always an improvisation, and the more digressive the better.

However, unlearning is a risky art. If art is controlled madness, then lack of control, and following pleasure, could take you anywhere. Could you be sure you wanted to go there, particularly with other people? Pleasure is an energy, and once you understand that it is a creative force – or rather, the creative force – you might begin to get somewhere. I saw that pop had always used this as a creed.

It would be a while before I realised that a series of indifferent relationships weren’t experimental. They weren’t even relationships. I was narrowing my mind, if not mortifying myself. And after studying Jarrett, I saw that some artists – particularly musicians like Prince, whom I would think about when I began my second novel The Black Album – never stopped producing.

But the idea of becoming more productive wasn’t it either, because I was beginning to see that although being an artist seemed to represent the ideal life – you follow your imagination and get paid, if not praised – it could never be sufficient. Talent can become an obstacle, and making art can become a way of retreating to a bunker where you would feel safe. I wondered if Jarrett had made that mistake with his work, and if he had learned from it.

Most of the time you have to be a person with another person, and if you’re lucky you can play with them, making demands and expecting demands back. New things emerge from this exchange. But facing other people straight on in their reality, with their odd, if not strange pleasures – and what, if anything, do they want? – could be too much. Racism, misogyny and other forms of inequality are intended to modify this impact by already diminishing the other. They come to you already filtered so you know what they are. Status is a form of protection, and equality the horror to be avoided. If speaking is a performance, then this form of improvisation is an attempt to find out what is unknown about oneself and others. And the unknown is, as the two Keiths knew, where the excitement lives.

Photograph © Daniela Silva

Letters from Prison