r - converting a row of data.frame to column names using data.table -


i have large data frame of 5 million rows, 3 columns. transform matrix has rows user_id, id columns, , value cnt. done melt , cast or 

xtabs(cnt ~ user_id + id, data = foo)

however object created large , following error 'dim' specifies large array

user_id id cnt 1      1.813e+14 21   1 2      1.559e+14 28   1 6      1.592e+14 71   2

i'm trying use data.table seams handle large data better data.frame, can't figure out how use data.table create contingency table want.
1 have idea how working? i'm thinking of creating , empty matrix appropriate dimensions , fill appropriate indexes.

try using built in data.frame co2 :

> xtabs(uptake ~ treatment + type, co2)             type treatment    quebec mississippi   nonchilled  742.0       545.0   chilled     666.8       332.1

or using tapply:

> with(co2, tapply(uptake, list(treatment, type), sum))            quebec mississippi nonchilled  742.0       545.0 chilled     666.8       332.1

and compare data.table:

> library(data.table) > > dt <- data.table(co2) > dt[, as.list(tapply(uptake, type, sum)), = treatment]     treatment quebec mississippi 1: nonchilled  742.0       545.0 2:    chilled  666.8       332.1

cautionary note: if same levels of type not appear in every treatment group not sufficient. in case necessary convert type factor in data table (as in co2).

added:

its possible rid of tapply , have pure data table approach this:

> dt[, setnames(as.list(.sd[,list(uptake = sum(uptake)), = type][, uptake]),  +   levels(type)), = treatment]     treatment quebec mississippi 1: nonchilled  742.0       545.0 2:    chilled  666.8       332.1

the cautionary note above applies here too.


Comments

Post a Comment

Popular posts from this blog

css - Text drops down with smaller window -

i've adapted css use in website , can't quite figure out why 1 aspect of page acting way acting. basically, when shrink window below size horizontally, of text "drops down". becomes issue if user viewing website through ipad vertical orientation. can view issue here . i'm hoping make if window gets smaller, placement of text remains intact. the problem area not have enough space put both side navigation , content on 1 line. make automatically adjust page's width when exceeds decrements of width. this code example, , highly unlikely work. if see css more clearly, provide exact code. @media , (max-width:960px) { /* assuming #wrapper 960px across */ #content {width:560px;} /* opposed 720px */ } edit the content drops down @ 980px, have shrink width after there. @media , (max-width:980px) { #container {width:520px;} }
Read more

php - Boolean search on database with 5 million rows, very slow -

i have script when user searches "apples", there coloumn related searches "shiny apples", , "rotten apples", etc. problem searches database has 5,600,000 rows , mysql lagging badly. here code: $get = mysql_query("select keyword searches match (keyword) against ('".mysql_real_escape_string(preg_replace('#([\s]+)#', '$1', $title))."' in boolean mode) limit 0,15"); this taking long time process , have active website searches getting performed every second, can recommend me tips? thanks! you need indexes. have had experiences in past creating indexes got significant performance gain.
Read more

c# - DetailsView in ASP.Net - How to add another column on the side/add a control in each row? -

i have detailsview control on page used edit various fields of record, works in respect. i looking way add 1 column (and if works, why not more) right, absolutely read-only, show same fields of record comparison purposes. i aware there no obvious way such thing out of box detailsview . have looked other controls (transposed gridview , recommended formview , listview ), nothing satisfies. have special data binding setup using detailsview , can't out of without losing features. anyone on how "hack in" additional columns (for display only) on detailsview ? the solution have now, use second detailsview , visible set false in aspx. in code, make sure databind hidden detailsview hosts data third column first, initial detailsview named itemdetails . and in item created event, pass third column html rendering of hidden controls (in last code block) : protected void itemdetails_itemcreated(object sender, eventargs e) { if (dataite...
Read more