regex - Write partly tab-delimited data to MySQL database -


i have mysql-database 7 columns (chr, pos, num, ia, ib, ic, id) , file contains 40 million lines each containing dataset. each line has 4 tab delimited columns, whereas first 3 columns contain data, , fourth column can contain 3 different key=value pairs separated semicolon

chr   pos   num   info 1     10203 3     ia=0.34;ib=nerv;ic=45;id=dskf12586 1     10203 4     ia=0.44;ic=45;id=dsf12586;ib=nerv 1     10203 5      1     10213 1     ib=nerv;ic=49;ia=0.14;id=dskf12586 1     10213 2     ia=0.34;ib=nerv;id=cap1486 1     10225 1     id=dscf12586

the key=value pairs in column info have no specific order. i'm not sure if key can occur twice (i hope not).

i'd write data database. first 3 columns no problem, extractiong values info-columns puzzles me, since key=value pairs unordered , not every key has in line. similar dataset (with ordered info-column) used java-programm in connection regular expressions, allowed me (1) check , (2) extract data, i'm stranded.

how can resolve task, preferably bash-script or directly in mysql?

you did not mention how want write data. below example awk shows how can each individual id , key in each line. instead of printf, can use own logic write data

[[bash_prompt$]]$ cat test.sh; echo "###########"; awk -f test.sh log {   if(length($4)) {     split($4,array,";");     print "in " $1, $2, $3;     for(element in array) {       key=substr(array[element],0,index(array[element],"="));       value=substr(array[element],index(array[element],"=")+1);       printf("found %s key , %s value %d line %s\n",key,value,nr,array[element]);     }   } } ########### in 1 10203 3 found id= key , dskf12586 value 1 line id=dskf12586 found ia= key , 0.34 value 1 line ia=0.34 found ib= key , nerv value 1 line ib=nerv found ic= key , 45 value 1 line ic=45 in 1 10203 4 found ib= key , nerv value 2 line ib=nerv found ia= key , 0.44 value 2 line ia=0.44 found ic= key , 45 value 2 line ic=45 found id= key , dsf12586 value 2 line id=dsf12586 in 1 10213 1 found id= key , dskf12586 value 4 line id=dskf12586 found ib= key , nerv value 4 line ib=nerv found ic= key , 49 value 4 line ic=49 found ia= key , 0.14 value 4 line ia=0.14 in 1 10213 2 found ia= key , 0.34 value 5 line ia=0.34 found ib= key , nerv value 5 line ib=nerv found id= key , cap1486 value 5 line id=cap1486 in 1 10225 1 found id= key , dscf12586 value 6 line id=dscf12586

Comments

Post a Comment

Popular posts from this blog

c# - DetailsView in ASP.Net - How to add another column on the side/add a control in each row? -

i have detailsview control on page used edit various fields of record, works in respect. i looking way add 1 column (and if works, why not more) right, absolutely read-only, show same fields of record comparison purposes. i aware there no obvious way such thing out of box detailsview . have looked other controls (transposed gridview , recommended formview , listview ), nothing satisfies. have special data binding setup using detailsview , can't out of without losing features. anyone on how "hack in" additional columns (for display only) on detailsview ? the solution have now, use second detailsview , visible set false in aspx. in code, make sure databind hidden detailsview hosts data third column first, initial detailsview named itemdetails . and in item created event, pass third column html rendering of hidden controls (in last code block) : protected void itemdetails_itemcreated(object sender, eventargs e) { if (dataite
Read more

javascript - firefox memory leak -

i have following script (i use raphaeljs also) draws circle, waits 1 second, draws circle again, etc (you can copy paste - functional, need include raphael raphaeljs.com): <div id="holder"> </div> <script src="raphael.js" type="text/javascript"></script> <script> var paper = raphael(10, 50, 320, 200); var rad = math.pi / 180; var finangle = 0; function sector(cx, cy, r, startangle, endangle, params) { var x1 = cx + r * math.cos(-startangle * rad), x2 = cx + r * math.cos(-endangle * rad), y1 = cy + r * math.sin(-startangle * rad), y2 = cy + r * math.sin(-endangle * rad); return ["m", cx, cy, "l", x1, y1, "a", r, r, 0, +(endangle - startangle > 180), 0, x2, y2]; } var path = sector(100,100,50,0,180); var arc = paper.path(path); function draw() { path = sector(100,100,50,0,finangle);
Read more

Trying to import CSV file to a SQL Server database using asp.net and c# - can't find what I'm missing -

i student , quite new programming, , given task using asp.net , c#, without being taught either. plan learn teach ourselves. the task stuck on making website import csv file sql server database of last year's room bookings around campus. the table has following columns: request_id; priority; module_id; day; start_time; length; park; students; room_code; status; semester_id; linked_request; week_1; week_2; week_3; week_4; week_5; week_6; week_7; week_8; week_9; week_10; week_11; week_12; week_13; week_14; week_15; ) in code, try make work 3 columns (request_id,priority,module) starters. when press import button, want read .csv file (with fixed name @ fixed directory) eg "1,2,3" - comma separating fields [delimiter] - , import sql server datab
Read more