Pallenberg is known for her romantic involvement with Rolling Stones band members Brian Jones, whom she met in 1965 in Munich, where she was working on a modelling assignment, and Keith Richards, for whom she left Jones in 1967 while on holiday in Morocco. She remained in a relationship with Richards until 1980 although they never married. There were rumours that she also had a brief affair with Mick Jagger during the filming of Performance, although Pallenberg denied the affair in March 2007 when Performance was released on DVD.
Pallenberg and Richards were together from 1967 - 1979. They had three children: son Marlon (born 10 August 1969), Angela (née Dandelion; born 17 April 1972) and a second son, Tara (26 March 1976- 6 June 1976), who died in his cot 10 weeks after his birth.