How do I get my table to sort +/- percentages in the correct order in dataset in R?

Consider the sample dataset measuring UK deprivation level by constituency with a percentage comparison with national average…

                                  ConstituencyName     c11Deprived   avgDeprived
1                                      Aberavon         0.46         -12.06%
2                                     Aberconwy         0.51          -2.31%
3                                Aberdeen North         0.76          43.86%
4                                Aberdeen South         0.45         -13.52%
5                            Airdrie and Shotts         0.87          65.35%
6                                     Aldershot         0.41         -21.40%

I created the avgDeprived variable by calculating a decimal proportion and used the label_percent() function to make it a percentage. Therefore it is a character variable However every time I try to sort the percentage by ascending order I get the following output…

                                  ConstituencyName     Deprived     avgDeprived
1                                      Aberavon         0.46         -12.06%
4                                Aberdeen South         0.45         -13.52%
2                                     Aberconwy         0.51          -2.31%
6                                     Aldershot         0.41         -21.40%
3                                Aberdeen North         0.76          43.86%
5                            Airdrie and Shotts         0.87          65.35%

When my desired output is…

                                ConstituencyName     Deprived     avgDeprived
6                                     Aldershot         0.41         -21.40%
4                                Aberdeen South         0.45         -13.52%
1                                      Aberavon         0.46         -12.06%
2                                     Aberconwy         0.51          -2.31%
3                                Aberdeen North         0.76          43.86%
5                            Airdrie and Shotts         0.87          65.35%

I have used gtools::mixedorder() to solve this problem which works but when I click another variable (i.e ConstiuencyName) to sort it and then click back on avgDeprived it goes back to the previous method of sorting.

Is there a simple way to fix this using R (preferably without having to change percentages back to numeric)?

#Data
df1 <- structure(list(ConstituencyName = structure(1:6, .Label = c("Aberavon", 
"Aberconwy", "Aberdeen North", "Aberdeen South", "Airdrie and Shotts", 
"Aldershot"), class = "factor"), c11Deprived. = c(0.46, 0.51, 
0.76, 0.45, 0.87, 0.41), avgDeprived = c("-12.06%", "-2.31%", 
"43.86%", "-13.52%", "65.35%", "-21.4%")), row.names = c(NA, 
-6L), class = "data.frame")

#Code
#Arrange
df2 <- df1[order(as.numeric(gsub('%','',df1$avgDeprived))),]

    ConstituencyName c11Deprived. avgDeprived
6          Aldershot         0.41      -21.4%
4     Aberdeen South         0.45     -13.52%
1           Aberavon         0.46     -12.06%
2          Aberconwy         0.51      -2.31%
3     Aberdeen North         0.76      43.86%
5 Airdrie and Shotts         0.87      65.35%