Hi Roger,
I gather you're aiming at the maximum distance at which
a planet can hold a satellite. That's a slightly different
issue than the one mentioned in the article Julian referenced.
That article has a rather nice description of the processes
that kept the sizes of the moons of gas giants down. (True,
Titan and Ganymede and friends are big objects, but they're
still small relative to Jupiter and Saturn, and this article
gives you some idea why there's no Earth-sized moon going
around Jupiter or a Mars-sized one around Saturn.)
The limit you're looking for is the "Hill Sphere" radius:
http://en.wikipedia.org/wiki/Hill_sphere
I gather that this limit is sort of a theoretical, upper
one. All known objects are well within the Hill radius, and
some numerical work indicates that the upper limit might be
brought in quite a bit, and that there might be different
limits for prograde and retrograde orbits (with the retrograde
ones having the higher limit).
-- Bill