Gray uses this word to refer to the soft, metallic sound that comes from the small bells around the necks of the herd of cattle. The soft noise coupled with the setting sun build on the dreamy and quiet scene that he has portrayed in the opening.
The verb "to toll" traditionally refers to causing a great bell to sound by pulling on a rope. Such bells are often associated with churches, which in turn are often near graveyards. Paired with the noun "knell" which refers to the sound that the large bell makes, this phrase has strong associations with such somber occasions as church services and funerals.
Toll, knell, parting: these three words in the first line of the poem establish the mood of the elegy as one of somber meditation, as Gray reminds readers of the inevitable presence of death as the final condition of humankind and sets the scene for his elegy.
