Adding suffix to filename during for loop in bash

Unix & Linux Asked by Mishal Ahmed on August 3, 2020

This has probably been asked but the existing answers don’t make sense to me. I am running a loop over multiple CSV files in a folder. This folder has non-csv files also and hence I must specify the csv files explicitly. I am modifying the csv files in several ways using awk, cut etc. After I am done, I want to redirect the output to new csv files with slightly modified names as follows:

TLC_2017.csv > TLC_2017_prepped.csv

Here’s a MWE:

for file in TLC*.csv; do
    cut -d, -f2- ${file} > ${file}_prepped

The problem is that I am getting new files with names such as TLC_2017.csv_prepped. In other words, the suffix is being added to the file extension and not the name. How do I add the suffix to the filename and not the extension?

If an answer to an existing question solves my problem, kindly don’t just link to it but also provide a little bit of explanation so that I can figure it out. Thanks!

One Answer

for file in TLC*.csv; do
    cut -d, -f2- "${file}" > "${file%.*}_prepped.csv"

${file%.*} removes everything after the last dot of $file. If you want to remove everything after the first dot, then you would use %%.

Likewise, ${file#*.} (${file##*.}) removes everything before the first (last) dot.

Read more in Parameter Expansion, Bash manual.

And remember, always quote your variables. You can use shellcheck to help you debugging your scripts. It would warn you about unquoted variables.

If looping over files with different extensions, the target file extension cannot be hardcoded as done above, so ${file##*.} is needed. As minimal example, which you can try in an empty test directory, this makes a _prepped copy to every file with an extension:

touch A.file.txt B.file.dat C_file.csv D_file.ko E
for file in *.*; do
    cp "$file" "${noext}_prepped.${ext}"

After execution,

$ ls
A.file_prepped.txt  A.file.txt  B.file.dat  B.file_prepped.dat
C_file.csv  C_file_prepped.csv  D_file.ko  D_file_prepped.ko  E

Correct answer by Quasímodo on August 3, 2020

Add your own answers!

Related Questions

How to expand “word” in bash? Not talking about file wildcards

2  Asked on October 31, 2021 by luis-santos


Parse a string using keywords

7  Asked on October 31, 2021 by beaker


Why does the command a-=2 fail?

4  Asked on October 31, 2021 by tmpbin


Shell dropping every other character of input

3  Asked on October 31, 2021 by michael-davis


Out of memory, but swap available

1  Asked on October 31, 2021


How can I display list of files in a directory?

4  Asked on October 31, 2021 by brocharvey


Booting from ISO suddenly stopped working

0  Asked on October 31, 2021 by ajgringo619


Does Pulseaudio leak memory?

2  Asked on October 31, 2021


Ask a Question

Get help from others!

© 2022 All rights reserved. Sites we Love: PCI Database, MenuIva, UKBizDB, Menu Kuliner, Sharing RPP