Your problem definition is unclear, but I would probably try setting this as an optimization problem where you try to optimize some definition of "smoothness" (e.g. minimize the forces/accelerations necessary to take your current position/velocity vector into a target position/velocity) rather than "hard-wiring" an abrupt transition between a straight line and a circular trajectory (which is always going to involve abrupt changes to the second-order derivative of the trajectory; i.e. a non-continuous acceleration)
Just to illustrate, this is what the a U-turn "optimally-smooth" trajectory would look given: a) the desired initial and final positions/velocities/accelerations; b) the desired total time of the trajectory; and c) an explicit definition of smoothness (in this case minimizing the sum of squares of the third-order derivatives of the trajectory; i.e. avoids discontinuities in acceleration): (see footnote below)
Now, let's say that instead of ending going straight down in the example above you would like to enter a circular trajectory at that point. For that, you simply need to constrain your final acceleration to lie perpendicular to the final velocity and scaled to match the desired target trajectory radius, and you end up with something like this:
So, coming back to your original question, how to transition between a straight trajectory and a semicircular one, it depends on a lot of things including your initial/target positions and velocities, whether you need to constrain the timing or not, whether you need to constrain the entry point or not, and whether there is a particular physically-meaningful characterization of "smoothing" that will be more appropriate for your problem. But, given all those things, you should be able to simply fit an optimization problem (or use some generalized solution like splines) to get your solution.
(footnote): For a general family of cost-functions, where "smoothness" is defined by continuity in the first n-th order derivatives, the solution trajectories are always a (n+1)-th order polynomial with respect of time, where you solve the polynomial parameters using your initial/end constrains by a simple linear regression (e.g. if you only constrain continuity in the initial and end positions, you get a straight line; if you additionally constrain continuity in the first-order derivatives -i.e. velocities- you get a quadratic trajectory, etc.) For a more complex cost function you can simply solve the appropriate optimization problem, using for example fminsearch