{"id":155,"date":"2014-04-03T19:51:06","date_gmt":"2014-04-03T16:51:06","guid":{"rendered":"http:\/\/tuijasonkkila.fi\/?p=155"},"modified":"2024-09-20T13:23:04","modified_gmt":"2024-09-20T10:23:04","slug":"network-once-again-now-with-yql","status":"publish","type":"post","link":"https:\/\/tuijasonkkila.fi\/?p=155","title":{"rendered":"Network once again, now with YQL!"},"content":{"rendered":"<p>While fiddling <a href=\"https:\/\/tuijasonkkila.fi\/2014\/04\/deconstructing-facebook-network\/\" target=\"_blank\" rel=\"noopener\">with the Facebook network, GEXF and JSON parsing<\/a> I remembered Yahoo! and its <a href=\"http:\/\/developer.yahoo.com\/yql\/guide\/running-chapt.html\" target=\"_blank\" rel=\"noopener\">YQL Web Services<\/a>. With it, you can get a JSON-formatted result from any, say, XML file out there. GEXF is XML.<\/p>\n<p>The YQL query language isn&#8217;t that handy if you are interested only in a selection of nodes; the XPath filter is only for HTML files, curiosly enough. I wanted the whole story though, so no problem. Here is how the YQL Console shows the result:<\/p>\n<p><a href=\"https:\/\/tuijasonkkila.fi\/wp-content\/uploads\/2014\/04\/yqlconsole.png\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/tuijasonkkila.fi\/wp-content\/uploads\/2014\/04\/yqlconsole-300x169.png\" alt=\"YQL Console\" class=\"alignnone size-medium wp-image-156\" width=\"300\" height=\"169\" srcset=\"https:\/\/tuijasonkkila.fi\/wp-content\/uploads\/2014\/04\/yqlconsole-300x169.png 300w, https:\/\/tuijasonkkila.fi\/wp-content\/uploads\/2014\/04\/yqlconsole-1024x577.png 1024w, https:\/\/tuijasonkkila.fi\/wp-content\/uploads\/2014\/04\/yqlconsole-624x351.png 624w, https:\/\/tuijasonkkila.fi\/wp-content\/uploads\/2014\/04\/yqlconsole.png 1445w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>With the REST query down below, you can e.g. transfer the JSON result to your local machine, in Unix like <code>curl 'http:\/\/query.yahooapis.com\/v1\/public\/yql?q=select%20*%20from%20xml%20where%20url%3D%22http%3A%2F%2Fusers.tkk.fi%2Fsonkkila%2Fnetwork%2Ffbmini.gexf%22&amp;format=json&amp;callback=' &gt; gexf.json<\/code><\/p>\n<p>The structure is more deep than in the JSON that the Cytoscape D3.js Exporter returns, but the only bigger change the D3 code needs is to have new references from the links\/edges to nodes.<\/p>\n<p>Like the documentation of <a href=\"https:\/\/github.com\/mbostock\/d3\/wiki\/Force-Layout#start\" target=\"_blank\" rel=\"noopener\">force.start()<\/a> says,<\/p>\n<blockquote><p>On start, the layout initializes various attributes on the associated nodes. The index of each node is computed by iterating over the array, starting at zero.<\/p><\/blockquote>\n<p>This is fine, if the <code>source<\/code> and <code>target<\/code> attributes in the edge array apply to this. Here, they do not. Instead, the attributes reference the <code>id<\/code> attribute in the respective nodes. So I needed to change that, and <a href=\"http:\/\/stackoverflow.com\/questions\/16824308\/d3-using-node-attribute-for-links-instead-of-index-in-array\" target=\"_blank\" rel=\"noopener\">excellent help was available<\/a>.<\/p>\n<p><a href=\"https:\/\/tuijasonkkila.fi\/tre\/fbyql.html\" target=\"_blank\" rel=\"noopener\">So far so good<\/a>, but using index numbers to access attribute values isn&#8217;t pretty and needs to be done differently. Maybe next time.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>While fiddling with the Facebook network, GEXF and JSON parsing I remembered Yahoo! and its YQL Web Services. With it, you can get a JSON-formatted result from any, say, XML file out there. GEXF is XML. The YQL query language isn&#8217;t that handy if you are interested only in a selection of nodes; the XPath &hellip; <a href=\"https:\/\/tuijasonkkila.fi\/?p=155\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Network once again, now with YQL!<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[149],"tags":[77,75,80,78,81,76,42],"class_list":["post-155","post","type-post","status-publish","format-standard","hentry","category-diverse-coding","tag-d3-js","tag-facebook","tag-force-directed-graph","tag-gephi","tag-gexf","tag-network-graph","tag-yql"],"_links":{"self":[{"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=\/wp\/v2\/posts\/155","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=155"}],"version-history":[{"count":9,"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=\/wp\/v2\/posts\/155\/revisions"}],"predecessor-version":[{"id":1021,"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=\/wp\/v2\/posts\/155\/revisions\/1021"}],"wp:attachment":[{"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=155"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=155"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tuijasonkkila.fi\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=155"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}