the ability to embed GPS location data into the EXIF data of image files can be very useful.
this can be done at the time the image is taken by certain cameras or once the images are downloaded onto a computer.
the main problem with connecting a GPS device to your camera is the problem of more cables to deal with - unless the GPS is built into the camera or via Bluetooth, then the delayed method is probably better, although more time consuming.
Taking photos with geotagging turned on such as with most smartphones default settings such as an iPhone, potentially creates security risks for you and your family if you post these online as it is very easy for anyone to locate on a map EXACTLY where you took these photos which maybe your home, work, child's school or playground.
TURN OFF geotagging for such photos you intend to post online on Facebook or Flickr or wherever else!