Koh Rong Dive Center, Koh Rong Travel Center, and many other boats go straight to Koh Rong Village from SihanoukVille or Koh Rong Samloem (the island next door).
This is a group of 30 restaurants and bungalows and rooms on the beach and hillside, going up into the jungle. It's about a 300 meter stretch, and it's growing.
There's a conservation charity, a bakery, and a few piers. Also, a dive center has their operations based on the pier.
Koh Rong Village is the "Happening" place on the island. You can rent fishing or diving boats here, or a kayak. A dozen places rent snorkeling gear. You can just jump off the pier for a swim.
A hundred meters from the village, starts the other 99% of Koh Rong Island, which is much quieter. We're talking about an island the size of Hong Kong Island here, so it'll be a few more weeks before the airport, golf course, and 7-11 set up shop here.
Rooms here cost from $3 for a dorm room, to $80 for a nice sized bungalow on the hillside, in the jungle, overlooking the bay. Most rooms are from $8 to $20.
Boats from the mainland (SihanoukVille) take 40 minutes, and most leave from the Serendipity Beach Pier. Round trips generally cost $20+, and leave SihanoukVille around 8am and 1pm. The trips from Koh Rong Island back to the mainland leave at 10am and 4pm.