<?xml version="1.0" encoding="UTF-8"?> <rss
version="2.0"
xmlns:content="http://purl.org/rss/1.0/modules/content/"
xmlns:wfw="http://wellformedweb.org/CommentAPI/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
> <channel><title>Cognitive Sandbox &#187; Unix</title> <atom:link href="http://cognitivesandbox.com/posts/tag/unix/feed/" rel="self" type="application/rss+xml" /><link>http://cognitivesandbox.com</link> <description>Vegan cooking, unix tidbits and other minor discoveries</description> <lastBuildDate>Sat, 03 Dec 2011 01:20:30 +0000</lastBuildDate> <language>en</language> <sy:updatePeriod>hourly</sy:updatePeriod> <sy:updateFrequency>1</sy:updateFrequency> <generator>http://wordpress.org/?v=3.3</generator> <item><title>Chained matching</title><link>http://cognitivesandbox.com/posts/chained-matching/</link> <comments>http://cognitivesandbox.com/posts/chained-matching/#comments</comments> <pubDate>Wed, 02 Dec 2009 01:03:07 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://www.cognitivesandbox.com/?p=2331</guid> <description><![CDATA[Sometimes you want to find references to &#8220;foo bar&#8221; or things like that, but regex won&#8217;t cut it. Maybe &#8220;foo&#8221; is on a different line from &#8220;bar&#8221;. It can get a little hairy when you want to chain that together even further. grep -ilR 'foo' * &#124; xargs grep -il 'bar' &#124; etc...]]></description> <content:encoded><![CDATA[<p>Sometimes you want to find references to &#8220;foo bar&#8221; or things like that, but regex won&#8217;t cut it. Maybe &#8220;foo&#8221; is on a different line from &#8220;bar&#8221;. It can get a little hairy when you want to chain that together even further.</p><p><code>grep -ilR 'foo' * | xargs grep -il 'bar' | <em>etc...</em></code></p> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/chained-matching/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item><title>Partitioning searching from processing</title><link>http://cognitivesandbox.com/posts/partitioning-searching-from-processing/</link> <comments>http://cognitivesandbox.com/posts/partitioning-searching-from-processing/#comments</comments> <pubDate>Thu, 30 Jul 2009 03:57:26 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://www.cognitivesandbox.com/?p=1676</guid> <description><![CDATA[There are two ways to grep recursively through all files of a certain type. grep -R --include='*.txt' 'search string' * find -type f -name '*.txt' -print0 &#124; \ xargs -0 grep 'search string' While grep&#8217;s include directive is certainly more concise, I&#8217;d argue that learning all of the general case is more useful in the [...]]]></description> <content:encoded><![CDATA[<p>There are two ways to grep recursively through all files of a certain type.</p><ul><li><pre><code>grep -R --include='*.txt' 'search string' *</code></pre></li><li><pre><code>find -type f -name '*.txt' -print0 | \
xargs -0 grep 'search string'</code></pre></li></ul><p>While grep&#8217;s include directive is certainly more concise, I&#8217;d argue that learning all of the general case is more useful in the long run. What if you want to exclude a certain path? What it you want to change file permissions instead of grepping? Combining find with xargs tends to be far more flexible than the alternatives.</p><h3>Examples</h3><ul><li><pre><code>find -type f -not \( -path '*/.svn/*' -a -prune \) \
-print0 | xargs -0 grep 'search string'</code></pre></li><li><pre><code>find -type f -print0 | xargs -0 du | sort -g | tail</code></pre></li></ul> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/partitioning-searching-from-processing/feed/</wfw:commentRss> <slash:comments>1</slash:comments> </item> <item><title>Symbolically merging one directory into another</title><link>http://cognitivesandbox.com/posts/symbolically-merging-one-directory-into-another/</link> <comments>http://cognitivesandbox.com/posts/symbolically-merging-one-directory-into-another/#comments</comments> <pubDate>Mon, 15 Jun 2009 18:30:48 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://www.cognitivesandbox.com/?p=1540</guid> <description><![CDATA[I found this technique necessary when working with vBulletin. I want to keep the core directory clean for easy upgrades (which often involves replacing the directory with the new release) and I don&#8217;t want to have to tediously copy files in each time. Solution: keep the files separate and recreate symbolic links on the fly. [...]]]></description> <content:encoded><![CDATA[<p>I found this technique necessary when working with vBulletin. I want to keep the core directory clean for easy upgrades (which often involves replacing the directory with the new release) and I don&#8217;t want to have to tediously copy files in each time. Solution: keep the files separate and recreate symbolic links on the fly.</p><pre><code># First remove any pre-existing links
find . -type l -print0 | xargs -0 rm;
# Then symbolically link dir2 into dir1
for file in `cd dir2; find`; do
	if [ ! -e dir1/$file ]; then
		depth=`echo $file | awk -F '/' '{print NF}'`;
		prefix=`perl -e 'print "../"x$ARGV[0]' $depth`;
		ln -s "$prefix"dir2/$file dir1/$file;
	fi
done
</code></pre>]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/symbolically-merging-one-directory-into-another/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item><title>Fingerprint of a path</title><link>http://cognitivesandbox.com/posts/fingerprint-of-a-directory/</link> <comments>http://cognitivesandbox.com/posts/fingerprint-of-a-directory/#comments</comments> <pubDate>Wed, 20 May 2009 20:59:48 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://www.cognitivesandbox.com/?p=565</guid> <description><![CDATA[I have a code base and something isn&#8217;t working in the way it should. I want to find out if the problem is in some corrupt data or if someone&#8217;s edited some files here and there. Unfortunately, the path isn&#8217;t under any kind of version control, so what do I do? Well, I can download [...]]]></description> <content:encoded><![CDATA[<p>I have a code base and something isn&#8217;t working in the way it should. I want to find out if the problem is in some corrupt data or if someone&#8217;s edited some files here and there. Unfortunately, the path isn&#8217;t under any kind of version control, so what do I do?</p><p>Well, I can download a fresh copy of the code base and compare, right? Here&#8217;s how you can do that.</p><p><code>find ./path -type f -print0 | xargs -0 md5sum > path.md5s;</code></p><p>Do the same to the fresh installation and diff the outputs. That should give you a reasonable feel for what&#8217;s changed, yeah? I don&#8217;t know if this is the best approach, so if you know a better way, please tell me.</p><p><strong>Update:</strong> Seems that <code>diff -urN path1 path2</code> will do it one better and show the contents of what changed. It doesn&#8217;t have the same power of pruning subdirectories that you&#8217;d expect from find, but in most cases it&#8217;s strictly better.</p> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/fingerprint-of-a-directory/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item><title>Unix tips</title><link>http://cognitivesandbox.com/posts/unix-tips/</link> <comments>http://cognitivesandbox.com/posts/unix-tips/#comments</comments> <pubDate>Sun, 19 Apr 2009 07:56:09 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://www.cognitivesandbox.com/?p=273</guid> <description><![CDATA[When I first started this blog, I had the intention of using it, at least in part, to publish little command-line tips I found useful. I suppose with commandlinefu.com up in full swing, that&#8217;s no longer necessary. I&#8217;m honestly surprised that site only came about within the last six months. Applying social aggregation to tips [...]]]></description> <content:encoded><![CDATA[<p>When I first started this blog, I had the intention of using it, at least in part, to publish little command-line tips I found useful. I suppose with <a
href="http://www.commandlinefu.com/commands/browse/sort-by-votes">commandlinefu.com</a> up in full swing, that&#8217;s no longer necessary.</p><p>I&#8217;m honestly surprised that site only came about within the last six months. Applying social aggregation to tips within a field seems obvious now. That&#8217;s the thing with good ideas, yeah?</p> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/unix-tips/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item><title>Export a database subset</title><link>http://cognitivesandbox.com/posts/export-a-database-subset/</link> <comments>http://cognitivesandbox.com/posts/export-a-database-subset/#comments</comments> <pubDate>Tue, 18 Nov 2008 22:30:32 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://dev.cognitivesandbox.com/?p=9</guid> <description><![CDATA[I ran into an interesting bit of code today. Sometimes a database gets too big and you only want the structure. In that situation, there&#8217;s the -d parameter. In other cases, you want not only the structure, but some of the data too. By using the &#8211;where parameter creatively, we can limit the number of [...]]]></description> <content:encoded><![CDATA[<p>I ran into an interesting bit of code today. Sometimes a database gets too big and you only want the structure. In that situation, there&#8217;s the -d parameter. In other cases, you want not only the structure, but some of the data too. By using the &#8211;where parameter creatively, we can limit the number of rows mysqldump returns.</p><p><code>mysqldump --where="true LIMIT X" schema > output.sql</code></p><p>Unfortunately, this doesn&#8217;t allow for any granularity in the number of rows returned. Still, &#8220;get me the database and at most 1000 rows from each table&#8221; can be helpful in certain situations.</p> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/export-a-database-subset/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item><title>The top family</title><link>http://cognitivesandbox.com/posts/the-top-family/</link> <comments>http://cognitivesandbox.com/posts/the-top-family/#comments</comments> <pubDate>Sat, 15 Nov 2008 19:34:22 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://dev.cognitivesandbox.com/?p=41</guid> <description><![CDATA[I&#8217;ve recently discovered a series of useful commands. Much like top, these commands provide real-time reports on various system statuses. Know them. Love them. iotop iftop mytop apachetop]]></description> <content:encoded><![CDATA[<p>I&#8217;ve recently discovered a series of useful commands. Much like <a
href="http://en.wikipedia.org/wiki/Top_(Unix)">top</a>, these commands provide real-time reports on various system statuses. Know them. Love them.</p><ul><li><a
href="http://guichaz.free.fr/iotop/">iotop</a></li><li><a
href="http://ex-parrot.com/~pdw/iftop/">iftop</a></li><li><a
href="http://jeremy.zawodny.com/mysql/mytop/">mytop</a></li><li><a
href="http://freshmeat.net/projects/apachetop/">apachetop</a></li></ul> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/the-top-family/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item><title>Like top, but for files</title><link>http://cognitivesandbox.com/posts/like-top-but-for-files/</link> <comments>http://cognitivesandbox.com/posts/like-top-but-for-files/#comments</comments> <pubDate>Fri, 10 Oct 2008 00:43:04 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://dev.cognitivesandbox.com/?p=3</guid> <description><![CDATA[I was restoring some 20GB of MySQL data today, but I&#8217;d forgotten to -v the command, leaving me to guess how far along I was in the process. To compensate for this, I threw together a quick script that acts a little like top, but for files. Observe. watch --differences -n 5 'df; ls -FlAt;' [...]]]></description> <content:encoded><![CDATA[<p>I was restoring some 20GB of MySQL data today, but I&#8217;d forgotten to -v the command, leaving me to guess how far along I was in the process. To compensate for this, I threw together a quick script that acts a little like <a
href="http://en.wikipedia.org/wiki/Top_command">top</a>, but for files. Observe.</p><p><code>watch --differences -n 5 'df; ls -FlAt;'</code></p><p>It basically says &#8220;keep printing out disk usage and recently modified files every 5 seconds until I say otherwise&#8221;. Run that inside of /var/lib/mysql/foo and you&#8217;re set.</p> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/like-top-but-for-files/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> <item><title>Execute a command on multiple servers</title><link>http://cognitivesandbox.com/posts/execute-a-command-on-multiple-servers/</link> <comments>http://cognitivesandbox.com/posts/execute-a-command-on-multiple-servers/#comments</comments> <pubDate>Mon, 06 Oct 2008 21:43:54 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://dev.cognitivesandbox.com/?p=20</guid> <description><![CDATA[It&#8217;s both amazing and informative to watch those with more skill than you, especially when you can look through logs to see what they did. The latest nugget of awesome I learned reboots a series of apache instances on separate servers. for i in `seq 1 10` do ssh instance$i rcapache2 restart; done It basically [...]]]></description> <content:encoded><![CDATA[<p>It&#8217;s both amazing and informative to watch those with more skill than you, especially when you can look through logs to see what they did. The latest nugget of awesome I learned reboots a series of apache instances on separate servers.</p><p><code>for i in `seq 1 10` do ssh instance$i rcapache2 restart; done</code></p><p>It basically says &#8220;connect to instance1, instance2 and so on and restart the apache server on each&#8221;. This approach could easily be modified for more complex tasks. That it&#8217;s done on one line is what catches my attention. Prior to seeing this done, I would have SSHed into each server by hand.</p> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/execute-a-command-on-multiple-servers/feed/</wfw:commentRss> <slash:comments>1</slash:comments> </item> <item><title>Finding large files</title><link>http://cognitivesandbox.com/posts/finding-large-files/</link> <comments>http://cognitivesandbox.com/posts/finding-large-files/#comments</comments> <pubDate>Wed, 01 Oct 2008 18:11:23 +0000</pubDate> <dc:creator>Brad Beattie</dc:creator> <category><![CDATA[Uncategorized]]></category> <category><![CDATA[Unix]]></category> <guid
isPermaLink="false">http://dev.cognitivesandbox.com/?p=18</guid> <description><![CDATA[In OSX, I can find what&#8217;s taking up my disk space using Disk Inventory X or GrandPerspective. That&#8217;s all well and good, but today I needed to get similar results over SSH. Observe. du -k --max-depth=X &#124; sort -r -g &#124; head -n Y It basically says &#8220;from the current directory, list everything at most [...]]]></description> <content:encoded><![CDATA[<p>In OSX, I can find what&#8217;s taking up my disk space using <a
href="http://www.derlien.com/">Disk Inventory X</a> or <a
href="http://grandperspectiv.sourceforge.net/">GrandPerspective</a>. That&#8217;s all well and good, but today I needed to get similar results over SSH. Observe.</p><p><code>du -k --max-depth=X | sort -r -g | head -n Y</code></p><p>It basically says &#8220;from the current directory, list everything at most X deep by kilobytes, sort it in descending order and only show me the top Y results&#8221;. With values like X=4 and Y=10, you&#8217;re bound to find the behemoths.</p><h3>Update</h3><p>Here&#8217;s another way of getting the same results, but with the option for further refinement.</p><p><code>find -size +1M -type f -print0 | xargs -0 ls -Ssh1 | head</code><br
/> <code>find -size +1M -exec du -sk {} \; | sort -nr | less</code></p> ]]></content:encoded> <wfw:commentRss>http://cognitivesandbox.com/posts/finding-large-files/feed/</wfw:commentRss> <slash:comments>0</slash:comments> </item> </channel> </rss>
