There is a paved route that you can take to avoid the gravel, but it is not well marked. At just over 15 miles out from the center of Carlton there is a road that cuts sharply back and up to the left, called Bald Mountain Road. There used to be signage, but is is no longer marked. There is another road that cuts off to the left about a mile before it that you can use as a heads-up reminder. You take Bald Mountain about 7-8 miles to Bible Creek Rd. (I believe it's called Testament Creek on Google Maps, maybe the name has changed?). Veer to the right on Bible Creek/Testament Creek and go another 7-8 miles until you re-join Nestucca River Rd.
I was planning on taking this route, but one of my riding friends was gassed, and there is a significant amount of climbing on this paved route. It's been years since I rode this, so no guarantees on the condition of the road. But it used to be pretty good.
Regarding the gravel section on Nestucca River Rd., I think its more like 2-miles long, but it is mostly very hard packed as Lynne indicated. Part of it I was able to maneuver at up to 20 mph on 23c tires, but it was mostly a 10-15 MPH bone rattler.