The President held an off-site meeting at the site of the future museum