At the annual Milken Institute Global Conference, the anxious talk was about tariffs and hopes for trade agreements and de-escalation.