Masking time-series for eXplainable AI

Why did my Machine Learning (ML) model make this decision? Is it paying attention to the right elements of the data we give it?