ruby on rails - How do I convert a Nokogiri statement into Mechanize for screen scraping? -


i'm trying use mechanize scape tags page. i've used nokogiri scrape them before, i'm trying combine them wider mechanize class. here nokogiri statement:

page = nokogiri::html(open(@model.url, "user-agent" => request.env['http_user_agent'])) @model.icons = page.css("link[rel='apple-touch-icon']").to_s

and here thought mechanize equivalent it's not working:

agent = mechanize.new page = agent.get(@model.url, "user-agent" => request.env['http_user_agent']) @model.icons = page.search("link[rel='apple-touch-icon']").to_s

the first 1 returns link tag expected <link rel="apple-touch-icon" etc etc..></link>. second statement returns blank string. if take to_s off end super long output. assume it's error or actual mechanize object or something.

link long output when not converting string: https://gist.github.com/eadam/5583541

without sample html it's difficult recreate problem, general information might you.

that "long output" inspect output of nokogiri::nodeset got when used search method. if search returns multiple nodes, or nodes have lots of children, inspect output can go on ways, but, that's should do.

css , search similar, in return nodeset. css assumes string passed in css accessor, while search more generic, , attempts figure out whether passed in css or xpath expression. if figures wrong odds bad pattern find match. can use at or search generic , let nokogiri figure out, or at_css, at_xpath or css , xpath respectively replace them. at derivations return first matching node, similar using search('some_path').first.

to_s turns nodeset representation of source passed in. prefer more explicit, using either to_xml, to_xhtml or to_html.

why don't output search css? don't know because can't test against html you're parsing. answering questions, data-processing, gigo situation.


Comments

Post a Comment

Popular posts from this blog

php - cannot display multiple markers in google maps v3 from traceroute result -

i'm newbie in php mysql , google maps v3. have problems in google maps v3. made traceroute web based , map traceroute's result in google maps v3 final project of college education. have try many tutorials didn't work. have tried tutorials in forum nothing worked. problem cannot display multiple markers got location data database. maybe because i'm newbie , not know how do. confuse looping show markers. here code <?php error_reporting(e_all ^ (e_notice)); ini_set('max_execution_time', 360); $enable_log_user = false; global $ip, $host_name, $host_ip; $host = @$_post['host']; $trace = @$_post['trace']; $self = $_server['php_self']; include("phpsqlajax_dbinfo.php"); $connection = mysql_connect ('127.0.0.1', $username, $password); if (!$connection) { die('not connected : ' . mysql_error());} $db_selected = mysql_select_db($database, $connection); if (!$db_selected) { die ('can\'t use db :
Read more

c# - DetailsView in ASP.Net - How to add another column on the side/add a control in each row? -

i have detailsview control on page used edit various fields of record, works in respect. i looking way add 1 column (and if works, why not more) right, absolutely read-only, show same fields of record comparison purposes. i aware there no obvious way such thing out of box detailsview . have looked other controls (transposed gridview , recommended formview , listview ), nothing satisfies. have special data binding setup using detailsview , can't out of without losing features. anyone on how "hack in" additional columns (for display only) on detailsview ? the solution have now, use second detailsview , visible set false in aspx. in code, make sure databind hidden detailsview hosts data third column first, initial detailsview named itemdetails . and in item created event, pass third column html rendering of hidden controls (in last code block) : protected void itemdetails_itemcreated(object sender, eventargs e) { if (dataite
Read more

javascript - firefox memory leak -

i have following script (i use raphaeljs also) draws circle, waits 1 second, draws circle again, etc (you can copy paste - functional, need include raphael raphaeljs.com): <div id="holder"> </div> <script src="raphael.js" type="text/javascript"></script> <script> var paper = raphael(10, 50, 320, 200); var rad = math.pi / 180; var finangle = 0; function sector(cx, cy, r, startangle, endangle, params) { var x1 = cx + r * math.cos(-startangle * rad), x2 = cx + r * math.cos(-endangle * rad), y1 = cy + r * math.sin(-startangle * rad), y2 = cy + r * math.sin(-endangle * rad); return ["m", cx, cy, "l", x1, y1, "a", r, r, 0, +(endangle - startangle > 180), 0, x2, y2]; } var path = sector(100,100,50,0,180); var arc = paper.path(path); function draw() { path = sector(100,100,50,0,finangle);
Read more