'The thread with which I used to tie her hair tight' or 'The thread in which I used to tie her hair tight'?

I don't think that "in which" is idiomatic in this semantic context; an ngram shows that it is never found (ngram) for the basic form, whereas "with which" is. It is safer to use this latter.

They're both grammatical.

I would use in which, because

the thread with which I used to tie her hair tight,

sounds too much like

the thread which I used to tied her hair tight,

which means that people could get confused as to the meaning. (In fact, if you look at the comments, people are getting confused by this.)