The only reason we need a driver is to stop serial console. Only try to load the driver in this case. Change-Id: I5a4e0a40659042087b7a466baae0a8c3dc136211