Watch, Ask and Adapt