Super celebrities, flaming basketballs, fighting fish - it's all in a day's work for director Sam Brown as he takes us behind the scenes of his latest promo shoot for Jay-Z track On To The Next One, shot through Flynn@DNA.

"I get a call about a Jay-Z track, and it turns out to be great. Raw, stripped down, back to basics. The images are already coming on the first listen. I usually find good ideas come instantly or not at all.

The track has this sample in it that sounds like a record being played backwards. It puts me in mind of some pretty dark imagery and I decide to take a big risk on pitching something pretty extreme.

So much high profile work is formulaic, mostly because it's very hard to get labels and artists to trust you on something that doesn't have instant commercial appeal. It's a shame. I think that many directors are far more interesting than the work they get to make. I'm helped here because this is the fifth single off the album, the stakes are low and Jay is pressing for something different that moves outside the narrow boundaries of hip-hop videos.

Jay loves the treatment, so I'm shooting it in LA only a week later.

I'm speaking with Regan Jackson, the art director, about which of the set-ups he thinks we can achieve on the limited budget we have. 'We're going to do all of them', he says, and then I don't see him for three days. Sure enough it's all there on the day, and more things I hadn't even asked for. Regan worked on all the multi million dollar Hype Williams videos back in the golden age, so I'm especially impressed at his resourcefulness.

Chris Probst, the DP, is amazing. If something needs moving he'll move it himself instead of wasting time asking someone else to do it. His lighting is impeccable. We talk about this heavy, kind of soapy look that I want everything to have. It's only when I'm putting the neg up in the TK a week later that I realise how brilliantly and subtly he's understood me: everyone's skin has this soft, cloudy quality to it. I'm always amazed at how tiny decisions can make a huge difference to how a film feels.

Matthias Koenigsweiser, a brilliant DP in his own right, is lighting the second unit, which is mostly table-top stuff. He and Bouha Kazmi, the second unit director, are having a fine time pouring cream and molasses on expensive props. There's a constant production line feeding them stuff, and many of the ideas are improvised on the spot.

Jay and his entourage arrive halfway through the first day. They will be here for just two hours. I meet Chaka, his lovely creative director, who leads me past a number of people to meet him in his Winnebago. It's interesting to meet high profile people in the flesh, but they are very rarely remarkable. They don't transmit any of their 'greatness'. But with Jay-Z it's impossible to ignore who he is. I feel slightly awed. We chat for a little bit, and it's clear that he's very charming and smart. I like him straight away. He has that not-much-eye-contact thing that only the super famous brand of celebrities has. The less famous are always wanting to catch someone's eye. We run out of things to talk about, and I slope off back to set.

When he walks on to set it's different, the dynamic changes. When you're directing people you can become very consumed in what you're doing, and you can lose sight of the way you would normally talk to someone. The natural order of things temporarily overturns. You have to speak directly to the subject and be honest about what you want, no matter who they are. I've found that artists at all levels really want to be directed. It's an extremely uncomfortable thing for someone, even someone who's made a hundred videos, to be in front of the camera and not know what to do.

I like it when a video is like a portrait, one in which you control the character. One of the great things about directing videos is that you can shape artists to how you think they should be. I've always liked Rick Rubin's approach to production, stripping away all the bullshit and putting people in touch with the basics. I've tried to do the same with videos. Jay's performance rolls off as sparse and raw and brooding. Perfect.

The stylist, Arielle, brings the dancing girls out to see me while I'm shooting some flaming basketballs. We're covering them in kerosene and dropping them from the ceiling. They look both astonishing and astonished, pressed into gothic garb with hair wound up into funny shrub-like shapes and faces covered in diamonds, like billion-dollar acne. We've shifted the shooting schedule so they have been hanging around all day and watching everything we're doing. They all look genuinely spooked, and huddle like frightened deer in a corner of the studio.

We're shooting some boxers. It becomes immediately apparent that one is much better than the other, and is afraid to hit him. These guys won't make the cut. We also shoot some fighting fish, but all they want to do is circle and sniff each other. I hate wasting footage, and very rarely shoot anything that doesn't make the edit. I imagine them telling their family and friends and tuning in to see it air, only to find they're not there. I always feel bad about this sort of thing.

All day the art dept have been constructing a paper wall made of thousands of post-it notes stuck to the white cove. I've been eyeing it constantly, worried that it might not work. It was an idea I had in passing that Regan leapt on and developed. I almost tried to talk him out of it again. It's always nice to do things on a really giant scale - but as the scale of things gets bigger the materials need to get cheaper and cheaper. I often find myself looking for interesting ways to use very inexpensive materials, but massively. There's a lot of beauty in plastic sheeting and paper and strip lights, I tend to use a lot of them. We blow the post-its with office fans and it looks beautiful, like funny sound waves.

I'm aware that as I'm shooting the images are getting darker and odder. Jay and his people came and went so they haven't seen anything else. I wonder what the hell they're going to think of it all. Whenever I've been given a big opportunity I've tried to be a bit reckless with it, to take chances. It's very rare that this doesn't pay off in some way. Playing things safe never gets you anywhere.

Back in London now, Amanda James is cutting the video. I have a few ideas about where things might go, but really it's a free-for-all, a giant puzzle. I tell her that I want the edit to feel quite brutal, and that we should be playing games with the lyrics and the images, trying to find the tension between them. I'm constantly amazed at how one image can feel so different depending on what is around it in the cut, and how it's meaning can change completely depending on where you put it.

I find with cutting it's pretty much 95 per cent pounding away at it, trying and retrying things. Every shot has to earn its place. Even in a video that is just visual, that doesn't mean anything, there needs to be a reason to leave something in. I'm always looking to take things out, to simplify as much as possible. But it's the last 5 per cent that makes the video sing, where the building blocks are there and a great editor really starts to do their job. Amanda cuts this beautifully, the edits jar and collide when they need to, or melt away when they don't. What she comes up with is almost hard to watch, which is exactly right.

I get an email that they love the offline, and that Jay wants to call to say thanks. I'm really excited to talk to him, but it pops up on the mobile while I'm bathing the kids so I decide I can't answer it. A week later the video airs on the New Year's Eve show with Carson Daly, to a massive TV audience. Immediately rumours begin to grow on the internet about Jay-Z being a Satanist, a Freemason, a member of the Illuminati (whoever they are)… or some mushy amalgamation of all three.

It's really strange to see your ideas scrutinized in this way, frame by frame. All the signs and meanings that you never intended. People are saying they can find faces in the thumbprints, that the ink is forming skulls, and that Jay is forming arcane signals with his hands. It's all brilliant fun."