The buff apears even if you aren't in the same map as the elite, i got the buff while doing bounties and a party member in a normal rift killed an elite.
The reverse also happened, he received the buff while fighting nothing @ all after i killed an elite.
Being in a different Grift floor as the killed elite also seems to be inrelevant.
Illusions also trigger the gem, allowing a "on demand" buff to be prepared for some1 in a greater rift.
How to exploit this for Grifts.
Create a game on any difficulty, search for a illusion elite, lure that to a waypoint.
Prepare a Grift stone and invite a friend to the party, who then joins the game, leaves (but stays in group) and whenever you lose your bane buff that person can log in and port back+leave game.
Free 20% damage in GR, regardless of if you killed an elite recently or not.