I recommend a driving route on the Old West Highway (26) that will take you over Mt Hood, through small towns along Highway 26 and along the scenic John Day River directly to Baker City. The total driving time is about 7 hours.
For your first night—if you’re up for driving—is the quaint town of Mitchell, Oregon, about 4 hours away from Portland. If you’d like to save the bulk of your driving and sightseeing for the next day, treat yourself to a night at Timberline Lodge on Mt. Hood.