Kanan change?

I fought a phoenix team today and as soon as I killed kanan his offense down disappeared from my character. Im pretty sure it would stay permanently (unless cleansed) if kanan was killed before the st...