Just before Xmas in 2017, having had a minor operation, I was lying in bed, bored, uncomfortable and in no mood to read, when for reasons I can’t explain, I thought I’d listen to Keith Jarrett’s The Köln Concert all through. I’d played it massively over the years, on record, cassette, CD and now on download. But since I tend to listen to music while doing something else, reading, writing or looking out of the window, I can’t say I’d actually heard or immersed myself in it for a long time.

I recalled that the double album was recorded live in Cologne in 1975 and a lot of people bought it, even those who would never have listened to anything quick or abstract by Charlie Parker or John Coltrane. Musicians – not novelists, movie or theatre directors or painters – had been at the centre of the culture I grew up in. They were our political and spiritual guides and we considered it crucial to keep in touch with what they were thinking. Despite this, for some unknown reason, I didn’t hear the album until around 1987.

I remember being depressed at the time because I told everyone so, and then I left London briefly, to stay with Karen, in Cardiff, Wales. In the early seventies we had been at college in Bromley together, doing our A levels, and she was the first female friend I’d had. It was not a romantic attachment: better educated and more cultured than me, she was right of centre then, argumentative and good fun to be around. She came often to the house and my father liked to talk with her. He encouraged us to start a magazine together, which had one issue.

In Cardiff, expecting her to welcome a definitive account of my depression, I arrived to discover that she had married a Buddhist and become one herself. They were wearing orange robes, burning incense and sitting still for long periods. I was appalled: how happy they were despite being ridiculous, having no drugs in the house and wearing a colour that did no one any favours. I grasped that not only would I receive little attention, but that afternoon they also wanted to see Raiders of the Lost Ark, which had just been released. Of course, its optimism hurled me into a blacker mood. A good Tarkovsky might have elevated my spirits.

My mood would turn darker that evening. Buddhists, according to my bias, tended to listen to chanting music or something somnolent suitable for aromatherapy. The new Thatcherite capitalism was depleting and exhausting people. If you couldn’t keep up, you could cross your legs and check out. Nietzsche called Buddhism ‘a kind of hygiene’. But I was half-asleep already; I wanted to wake up and tune in to the cosmic meaning. That was why I was there.

Karen’s husband, whom I had taken against – particularly after the devoted way I saw him hold and caress her foot when we visited a shoe shop – put on The Köln Concert, which I knew nothing about. When he informed me that the piece was an improvisation I suspected my visit would be short. In the seventies and early eighties I had worked in the theatre and at that time directors liked to use improvisation to ‘free the actors up’. These exercises were interminable; I had never seen an actor achieve anything through improvisation that wouldn’t have been better coming from a writer who had thought about what he was doing. But, having no choice – ‘the ears have no lids’, as Jacques Lacan reminds us – I listened. And those first five notes knocked me out; it was like receiving five firm blows to the head.

That night we had supper and a young woman friend of theirs came round. When the woman had gone and the Buddhists were in bed I retired to the attic where I was staying, and waited. In those days I was an enthusiast of one-night stands, where a never-repeated intensity and strangeness might occur with an unknown person who would remain mostly unknown – apart from their obscenity. Hoping for what Robert Stoller calls ‘reciprocated pathologies’, the couple would become and remain a kind of living fantasy for one another. That was the idea.

Earlier, the girl and I had whispered together. Now she came back. We lit candles; I crept downstairs for the record. We lay on the bed together and played it all night.

I curled up. Everything was wrong with me. Suffering from lack of curiosity, I was too fearful and inhibited to have sex with her, if that was what she wanted. Or, indeed, if it was what I wanted. Sex rarely lacks trauma: it is almost always shattering and there are few insignificant sexual encounters, however fleeting.

But I lacked the sense or ability to inform her of how I felt. If speaking is obviously the most important thing we do, I could have tried that. She might, after all, have wanted to hear and respond.

Certainly I had always considered it more profitable and interesting to talk with men than with women. My father and I had been close, and I had witnessed how much he and his brothers liked to talk. In contrast, my mother avoided social situations and conversation. She was already nervous, if not frightened and trapped. Later I understood that the freedom to speak, joke or tease could never be enjoyable for her. Petrified, secluded and busy trying not to go mad, more talk would only disorientate her. She was enigmatic to us, and appeared to have little idea of what was going on inside herself. Not that she wanted to be helped. She didn’t think it was necessary that people be interesting, funny or attractive. On the rare occasions when people came by to see us I wanted them to have a nice time and like us. Once, when I was enumerating the qualities of someone I liked, she interrupted to say, ‘Why can’t people just be nice?’ My analyst said that that was a profound remark. Clearly, he was on to something. He’s nice himself, but I can’t imagine that people pay him for only that.

Yet I had always been fascinated with women’s bodies, their gestures, clothes, voices and who they were. But, that night, what I called depression was rigidity and repetition; a taste of bitter nothingness. I was lost and afraid in a dark wood with no capacity to enjoy my own thoughts or those of anyone else. I could enter a tunnel of all-debasing, tantrumy fury where things would get dirty in my head, and I’d be tempted to throw myself under a train. Who doesn’t know someone who has killed themselves, and even admired their courage? In these moods you can forget that you are the engine of your own tempest.

The nearly dead certainly lack a sense of humour. The British child psychoanalyst Donald Winnicott calls depression ‘smoke over the battlefield’, but where exactly was the conflict taking place? Who was the speechlessness – the block, the dire shortage of words – for? How do you begin to understand what is going on inside you? I’d travelled a bit and recognised that we lived – if you were in London, if you’d benefited from the welfare state and were in work – in a relatively free society where the bright new individualism of the sixties was still being celebrated, albeit in a darker form. With me, the sources of oppression were within. I was exerting a reign of terror over myself, while destroying my ability to resist. I had put myself on my back.

Stephen Frears and I had made My Beautiful Laundrette and Sammy and Rosie Get Laid. I had some money for the first time and not long before had been at the Oscars, sitting next to Bette Davis, who had been kind. Now I was at a crossroads, with no direction home or ahead. I knew I should begin the novel I had been attempting since my teens. It would become The Buddha of Suburbia, but I didn’t know how to start. I couldn’t find the right voice for it. Or a voice for myself.

Having successfully sabotaged the Buddhist couple’s weekend and more or less lost Karen as a friend, back in London I bought The Köln Concert on cassette because I needed to know it better.

Letters from Prison