python - Optimize shell script (bash) to impove performance -

i have bash script use process text file:

#/bin/bash  dos2unix sourcefile.txt  cat sourcefile.txt | grep -v '\/' | grep -v '\-\-' | grep -v '#' | grep '[a-za-z]\*' > modified_sourcefile.txt  mv modified_sourcefile.txt sourcefile.txt # # read sourcefile file 1 line line , iterate... #  while read line   echo $line | grep -v '\/' | grep -v '\-\-' | grep -v '#'  if [ $? -eq 0 ]      # echo "current line " $line ";"     char1=`echo ${line:0:1}`    # echo "1st char " $char1    if [ -n "$char1" ]    # if blank-line, neglect line.             # echo "test passed"         var1=`echo $line | cut -d '*' -f 1`     var2=`echo $line | cut -d '*' -f 1`     var3=`echo $line | cut -d - -f 1`         var4=`echo $line | cut -d '*' -f 1`         var5=`echo $line | cut -d '*' -f 2`         var6=`echo $line | cut -d - -f 1`         var7=`echo $line | cut -d '*' -f 3 `           table1sql="insert ignore table1 (id,name,active_yesno,category,description,            last_modified_by,last_modified_date_time) select ifnull(max(id),0)+1,'$var1',1,            '$var2','$var3','admin',now() table1;"      echo $table1sql >> result.txt       privsql="insert ignore table2 (id,name,description,active_yesno,group_code,              last_modified_by,last_modified_date_time) select ifnull(max(id),0)+1,'$var1',          '$var3',1,'$var2','admin',now() table2;"      echo $privsql >> result.txt            table1privmapsql="insert ignore table1_table2_map (id,table1_id,table2_id,                   last_modified_by,last_modified_date_time) select ifnull(max(id),0)+1,                   (select id table1 name='$var1'),(select id table2 name='$var1'),'admin',now() table1_table2_map;"     echo $table1privmapsql >> result.txt          privgroupsql="insert ignore table2_group (id,name,category,active_yesno,last_modified_by,                       last_modified_date_time) select ifnull(max(id),0)+1,'tablegrp','$pgpcode',1,'admin',now() table2_group;"          echo $privgroupsql >> result.txt       privprivgrpsql="insert ignore table2_table2group_map (id,table2_id,table2_group_id,                         last_modified_by,last_modified_date_time) select ifnull(max(id),0)+1,                         (select id table2 name='$var1'),(select id table2_group name='tablegrp'),'admin',now() table2_table2group_map;"         echo $privprivgrpsql >> result.txt                    rolesql="insert ignore role (id,name,active_yesno,security_domain_id,last_modified_by,last_modified_date_time)                   select (select ifnull(max(id),0)+1 role),'$rolename',1, sd.id ,'admin',now()                   security_domain sd sd.name = 'general';"          echo $rolesql >> result.txt      fi                    fi                         done < "sourcefile.txt" 

the thing sourcefile.txt has on 11000 lines. takes 25 min complete :-( .

is there better way of doing it?

contents of sourcefile.txt:

aaa-something*location-some_where*abc 

to make script faster must minimize calls external commands , use bash possible.

1. read this article know useless use of commands.

2. read this article know how use bash manipulate strings.

3. replace repeating values(var1, var2, var4) assignment single value.

while optimizing cut can replace

var1=`echo $line | cut -d '*' -f 1` 

to

var1="${line%%\**}" 

and

var5=`echo $line | cut -d '*' -f 2` 

to

var5="${line%\**}" var5="${var5##*\*}" 

maybe not human-readable, works faster cut.

also

 echo $line | grep -v '\/' | grep -v '\-\-' | grep -v '#' 

can replaced that:

 if [[ "$line" =~ ([/#]|--) ]]; :; else      # code inside "if [ $? -eq 0 ]"  fi