How to delete an image that is currently being used in a process

So. I have an image that can be set as the background image on my vb app and when the user wants to change the image, I have it so that it fetches their chosen image from a showdialog and puts it into a specific filepath for the rest of the program to access. But, when the process is done more than once on its initial run (meaning there is no image in the directory yet) it gives and error saying the process cannot be completed due to ("C:userdata" & ProteusLogin.txtUsername.Text & "" & "backgroundimage.jpg", "delete.jpg") process being in use.

Here is the code.

Private Sub RDBCustom_doubleclick(sender As Object, e As EventArgs) Handles RDBCustom.Click
        RBLight.Checked = False
        RBOriginal.Checked = False
        RBDark.Checked = False
        Dim openfiledialog1 As New OpenFileDialog
        Try
            My.Computer.FileSystem.CopyFile(openfiledialog1.FileName, "C:userdata" & ProteusLogin.txtUsername.Text & "" & "backgroundimage.jpg")
        Catch
            If System.IO.File.Exists("C:userdata" & ProteusLogin.txtUsername.Text & "" & "backgroundimage.jpg") = True Then
                My.Computer.FileSystem.RenameFile("C:userdata" & ProteusLogin.txtUsername.Text & "" & "backgroundimage.jpg", "delete.jpg")
                System.IO.File.Delete("C:userdata" & ProteusLogin.txtUsername.Text & "" & "delete.jpg")
                My.Computer.FileSystem.CopyFile(openfiledialog1.FileName, "C:userdata" & ProteusLogin.txtUsername.Text & "" & "backgroundimage.jpg")
            End If

        End Try
    End Sub