AnswerBun.com

How to assign environment variables in parallel in bash

Unix & Linux Asked by kag0 on November 30, 2020

I’m trying to set several environment variables with the results from command substitution. I want to run the commands in parallel with & and wait. What I’ve got currently looks something like

export foo=`somecommand bar` &
export fizz=`somecommand baz` &
export rick=`somecommand morty` &
wait

But apparently when using & variable assignments don’t stick. So after the wait, all those variables are unassigned.

How can I assign these variables in parallel?

UPDATE: Here’s what I ended up using based off the accepted answer

declare -a data
declare -a output
declare -a processes

var_names=(
    foo
    fizz
    rick
)

for name in "${var_names[@]}"
do
    processes+=("./get_me_a_value_for $name")
done

index=0
for process in "${processes[@]}"; do
    output+=("$(mktemp)")
    ${process} > ${output[$index]} &
    index=$((index+1))
done
wait

index=0
for out in "${output[@]}"; do
    val="$(<"${out}")"
    rm -f "${out}"

    export ${var_names[index]}="$val"

    index=$((index+1))
done

unset data
unset output
unset processes

2 Answers

After some ruminations, I came up with an ugly workaround:

#!/bin/bash
proc1=$(mktemp)
proc2=$(mktemp)
proc3=$(mktemp)

/path/to/longprocess1 > "$proc1" &
pid1=$!
/path/to/longprocess2 > "$proc2" &
pid2=$!
/path/to/longprocess3 > "$proc3" &
pid3=$!

wait "$pid1" "$pid2" "$pid3"
export var1="<("$proc1")"
export var2="<("$proc2")"
export var3="<("$proc3")"
rm -f "$proc1" "$proc2" "$proc3"

As requested in a comment, here is how to make this more extensible for an arbitrarily large list:

#!/bin/bash
declare -a pids
declare -a data
declare -a output
declare -a processes

# Generate the list of processes for demonstrative purposes
processes+=("/path/to/longprocess1")
processes+=("/path/to/longprocess2")
processes+=("/path/to/longprocess3")

index=0
for process in "${processes[@]}"; do
    output+=("$(mktemp")
    $process > ${output[$index]} &
    pids+=("$!")
    index=$((index+1))
done
wait ${pids[@]}
index=0
for process in "${processes[@]}"; do
    data+="$(<"${output[index]}")"
    rm -f "${output[index]}"
    index=$((index+1))
done
export data

The resultant output will be in the data array.

Correct answer by DopeGhoti on November 30, 2020

If you have more jobs than can safely be run in parallel at the same time, you can use parset from GNU Parallel:

parset foo,fizz,rick somecommand ::: bar baz morty
export foo
export fizz
export rick

See details: https://www.gnu.org/software/parallel/parset.html

Answered by Ole Tange on November 30, 2020

Add your own answers!

Related Questions

All processes running on the same core

1  Asked on December 24, 2020 by roofkiller

 

capturing yum update transaction to replicate on another machine

0  Asked on December 23, 2020 by dan-schneider

     

Add Support CRAM-MD5 or DIGEST-MD5 to SMTP server

0  Asked on December 22, 2020 by alpav99

         

Linux delete lines from ini-file where key has no value

1  Asked on December 21, 2020 by sunwave121

     

What caused traffic on port 631?

0  Asked on December 21, 2020 by u20200411

   

Bind/Unbind Linux Module Automatically

1  Asked on December 20, 2020 by vladius

       

ls everything that is _not_ a symlink

1  Asked on December 20, 2020

     

Netgear39 Removal

0  Asked on December 20, 2020 by danny-bender

     

Join two csv files by matching columns, join command

1  Asked on December 20, 2020 by cdxun

   

Arithmetic on values with memory size units

4  Asked on December 20, 2020 by malan

   

case while option not showing last argument in shell script

1  Asked on December 19, 2020 by user2358844

   

Debian update to Buster, iptables throwing an error in

1  Asked on December 19, 2020 by jademalo

     

Logitech Marble Mouse – Linux Scroll Modifier Setup

3  Asked on December 18, 2020 by peter-l

   

ZFS setup, adding drives on the go

0  Asked on December 18, 2020 by timyorgut

         

Linux Font “Lato” is not available to the JVM

0  Asked on December 18, 2020 by valene

   

Ask a Question

Get help from others!

© 2022 AnswerBun.com. All rights reserved. Sites we Love: PCI Database, MenuIva, UKBizDB, Menu Kuliner, Sharing RPP, SolveDir