Monday, January 27, 2014

Bash Shell: Replace a String With Another String In All Files Using sed and Perl -pie Options

How do I replace a string with another string in all files? For example, ~/foo directory has 100s of text file and I'd like to find out xyz string and replace with abc. I'd like to use sed or any other tool to replace all occurrence of the word.

sed replace word / string syntax
The sed command is designed for this kind of work i.e. find and replace strings or words from a text file under Apple OX, *BSD, Linux, and UNIX like operating systems. The perl can be also used as described below.
The syntax is as follows:
sed -i 's/old-word/new-word/g' *.txt
GNU sed command can edit files in place (makes backup if extension supplied) using the -i option. If you are using an old UNIX sed command version try the following syntax:
sed 's/old/new/g' input.txt > output.txt
You can use old sed syntax along with bash for loop:
#!/bin/bash
OLD="xyz"
NEW="abc"
DPATH="/home/you/foo/*.txt"
BPATH="/home/you/bakup/foo"
TFILE="/tmp/out.tmp.$$"
[ ! -d $BPATH ] && mkdir -p $BPATH || :
for f in $DPATH
do
  if [ -f $f -a -r $f ]; then
    /bin/cp -f $f $BPATH
   sed "s/$OLD/$NEW/g" "$f" > $TFILE && mv $TFILE "$f"
  else
   echo "Error: Cannot read $f"
  fi
done
/bin/rm $TFILE

Tuesday, January 21, 2014

15 Practical Grep Command Examples In Linux / UNIX

In this article let us review 15 practical examples of Linux grep command that will be very useful to both newbies and experts.
First create the following demo_file that will be used in the examples below to demonstrate grep command.
$ cat demo_file
THIS LINE IS THE 1ST UPPER CASE LINE IN THIS FILE.
this line is the 1st lower case line in this file.
This Line Has All Its First Character Of The Word With Upper Case.

Two lines above this line is empty.
And this is the last line.

1. Search for the given string in a single file

The basic usage of grep command is to search for a specific string in the specified file as shown below.
Syntax:
grep "literal_string" filename
$ grep "this" demo_file
this line is the 1st lower case line in this file.
Two lines above this line is empty.

2. Checking for the given string in multiple files.

Syntax:
grep "string" FILE_PATTERN

This is also a basic usage of grep command. For this example, let us copy the demo_file to demo_file1. The grep output will also include the file name in front of the line that matched the specific pattern as shown below. When the Linux shell sees the meta character, it does the expansion and gives all the files as input to grep.
$ cp demo_file demo_file1

$ grep "this" demo_*
demo_file:this line is the 1st lower case line in this file.
demo_file:Two lines above this line is empty.
demo_file:And this is the last line.
demo_file1:this line is the 1st lower case line in this file.
demo_file1:Two lines above this line is empty.
demo_file1:And this is the last line.

3. Case insensitive search using grep -i

Syntax:
grep -i "string" FILE

This is also a basic usage of the grep. This searches for the given string/pattern case insensitively. So it matches all the words such as “the”, “THE” and “The” case insensitively as shown below.
$ grep -i "the" demo_file
THIS LINE IS THE 1ST UPPER CASE LINE IN THIS FILE.
this line is the 1st lower case line in this file.
This Line Has All Its First Character Of The Word With Upper Case.
And this is the last line.

4. Match regular expression in files

Syntax:
grep "REGEX" filename

This is a very powerful feature, if you can use use regular expression effectively. In the following example, it searches for all the pattern that starts with “lines” and ends with “empty” with anything in-between. i.e To search “lines[anything in-between]empty” in the demo_file.
$ grep "lines.*empty" demo_file
Two lines above this line is empty.
From documentation of grep: A regular expression may be followed by one of several repetition operators:
  • ? The preceding item is optional and matched at most once.
  • * The preceding item will be matched zero or more times.
  • + The preceding item will be matched one or more times.
  • {n} The preceding item is matched exactly n times.
  • {n,} The preceding item is matched n or more times.
  • {,m} The preceding item is matched at most m times.
  • {n,m} The preceding item is matched at least n times, but not more than m times.

5. Checking for full words, not for sub-strings using grep -w

If you want to search for a word, and to avoid it to match the substrings use -w option. Just doing out a normal search will show out all the lines.

The following example is the regular grep where it is searching for “is”. When you search for “is”, without any option it will show out “is”, “his”, “this” and everything which has the substring “is”.
$ grep -i "is" demo_file
THIS LINE IS THE 1ST UPPER CASE LINE IN THIS FILE.
this line is the 1st lower case line in this file.
This Line Has All Its First Character Of The Word With Upper Case.
Two lines above this line is empty.
And this is the last line.

The following example is the WORD grep where it is searching only for the word “is”. Please note that this output does not contain the line “This Line Has All Its First Character Of The Word With Upper Case”, even though “is” is there in the “This”, as the following is looking only for the word “is” and not for “this”.
$ grep -iw "is" demo_file
THIS LINE IS THE 1ST UPPER CASE LINE IN THIS FILE.
this line is the 1st lower case line in this file.
Two lines above this line is empty.
And this is the last line.

6. Displaying lines before/after/around the match using grep -A, -B and -C

When doing a grep on a huge file, it may be useful to see some lines after the match. You might feel handy if grep can show you not only the matching lines but also the lines after/before/around the match.

Please create the following demo_text file for this example.
$ cat demo_text
4. Vim Word Navigation

You may want to do several navigation in relation to the words, such as:

 * e - go to the end of the current word.
 * E - go to the end of the current WORD.
 * b - go to the previous (before) word.
 * B - go to the previous (before) WORD.
 * w - go to the next word.
 * W - go to the next WORD.

WORD - WORD consists of a sequence of non-blank characters, separated with white space.
word - word consists of a sequence of letters, digits and underscores.

Example to show the difference between WORD and word

 * 192.168.1.1 - single WORD
 * 192.168.1.1 - seven words.

6.1 Display N lines after match

-A is the option which prints the specified N lines after the match as shown below.
Syntax:
grep -A  "string" FILENAME

The following example prints the matched line, along with the 3 lines after it.
$ grep -A 3 -i "example" demo_text
Example to show the difference between WORD and word

* 192.168.1.1 - single WORD
* 192.168.1.1 - seven words.

6.2 Display N lines before match

-B is the option which prints the specified N lines before the match.
Syntax:
grep -B  "string" FILENAME

When you had option to show the N lines after match, you have the -B option for the opposite.
$ grep -B 2 "single WORD" demo_text
Example to show the difference between WORD and word

* 192.168.1.1 - single WORD

6.3 Display N lines around match

-C is the option which prints the specified N lines before the match. In some occasion you might want the match to be appeared with the lines from both the side. This options shows N lines in both the side(before & after) of match.
$ grep -C 2 "Example" demo_text
word - word consists of a sequence of letters, digits and underscores.

Example to show the difference between WORD and word

* 192.168.1.1 - single WORD

7. Highlighting the search using GREP_OPTIONS

As grep prints out lines from the file by the pattern / string you had given, if you wanted it to highlight which part matches the line, then you need to follow the following way.

When you do the following export you will get the highlighting of the matched searches. In the following example, it will highlight all the this when you set the GREP_OPTIONS environment variable as shown below.
$ export GREP_OPTIONS='--color=auto' GREP_COLOR='100;8'

$ grep this demo_file
this line is the 1st lower case line in this file.
Two lines above this line is empty.
And this is the last line.

8. Searching in all files recursively using grep -r

When you want to search in all the files under the current directory and its sub directory. -r option is the one which you need to use. The following example will look for the string “ramesh” in all the files in the current directory and all it’s subdirectory.
$ grep -r "ramesh" *

9. Invert match using grep -v

You had different options to show the lines matched, to show the lines before match, and to show the lines after match, and to highlight match. So definitely You’d also want the option -v to do invert match.

When you want to display the lines which does not matches the given string/pattern, use the option -v as shown below. This example will display all the lines that did not match the word “go”.
$ grep -v "go" demo_text
4. Vim Word Navigation

You may want to do several navigation in relation to the words, such as:

WORD - WORD consists of a sequence of non-blank characters, separated with white space.
word - word consists of a sequence of letters, digits and underscores.

Example to show the difference between WORD and word

* 192.168.1.1 - single WORD
* 192.168.1.1 - seven words.

10. display the lines which does not matches all the given pattern.

Syntax:
grep -v -e "pattern" -e "pattern"
$ cat test-file.txt
a
b
c
d

$ grep -v -e "a" -e "b" -e "c" test-file.txt
d

11. Counting the number of matches using grep -c

When you want to count that how many lines matches the given pattern/string, then use the option -c.
Syntax:
grep -c "pattern" filename
$ grep -c "go" demo_text
6

When you want do find out how many lines matches the pattern
$ grep -c this demo_file
3

When you want do find out how many lines that does not match the pattern
$ grep -v -c this demo_file
4

12. Display only the file names which matches the given pattern using grep -l

If you want the grep to show out only the file names which matched the given pattern, use the -l (lower-case L) option.

When you give multiple files to the grep as input, it displays the names of file which contains the text that matches the pattern, will be very handy when you try to find some notes in your whole directory structure.
$ grep -l this demo_*
demo_file
demo_file1

13. Show only the matched string

By default grep will show the line which matches the given pattern/string, but if you want the grep to show out only the matched string of the pattern then use the -o option.

It might not be that much useful when you give the string straight forward. But it becomes very useful when you give a regex pattern and trying to see what it matches as
$ grep -o "is.*line" demo_file
is line is the 1st lower case line
is line
is is the last line

14. Show the position of match in the line

When you want grep to show the position where it matches the pattern in the file, use the following options as
Syntax:
grep -o -b "pattern" file
$ cat temp-file.txt
12345
12345

$ grep -o -b "3" temp-file.txt
2:3
8:3

Note: The output of the grep command above is not the position in the line, it is byte offset of the whole file.

15. Show line number while displaying the output using grep -n

To show the line number of file with the line matched. It does 1-based line numbering for each file. Use -n option to utilize this feature.
$ grep -n "go" demo_text
5: * e - go to the end of the current word.
6: * E - go to the end of the current WORD.
7: * b - go to the previous (before) word.
8: * B - go to the previous (before) WORD.
9: * w - go to the next word.
10: * W - go to the next WORD.

Sunday, January 19, 2014

How Google's smart contact lens works

By : HAYLEY TSUKAYAMA

Wearable devices are already bringing technology much closer to you than you ever may have expected, but Google's just kicked it up to a whole new level.

The company has announced a project to make a smart contact lens. But this gadget isn't going to be used to deliver your email straight into your skull - at least not yet. This project is working to tackle one of the biggest health problems facing the country today: diabetes.

Given the wariness around wearable devices and their capabilities for data collection, the idea that the company would get that much closer raises the question: how will Google handle this data? Or, for that matter, how can any commercial company stepping into a new world of collecting sensitive medical data deal with the security concerns?

It's a question that Google's clearly thought a lot about, said Joseph Lorenzo Hall, chief technologist at the Center for Democracy and Technology, who was briefed on the lens before the company's Thursday announcement. Hall said that Google assured him that the data would not be added to the company's banks of personal information gathered from other servers.

"The data will never hit Google's servers," he said. "That's a forward-thinking affirmative claim that they're making. That is important."

The soft contact lens that Google's is introducing - it's still just a prototype - houses a sensor between two layers of lenses that measures the glucose levels in tears. A tiny pinhole in the lens lets tear fluid seep over the glucose monitor to get regular readings. Right now, the company said, it can get a level reading once every second. The lens also features a tiny antenna, capacitor and controller, so that the information gathered from the lens can move from your eye to a device such as a handheld monitor, where that data can be read and analyzed. It will draw its power from that device and communicate with it using a wireless technology known as RFID.

Given the sensitive nature of the data, Hall said, Google has also said that it will make sure any data transferred from the lens will be insulated against anyone who might want to change its readings - something that could have potentially fatal consequences if patients inject the wrong amount of insulin. Google has also worked to build in safeguards against other kinds of problems, such as a piece that's a little like a circuit-breaker to prevent the lens from overheating.

The National Diabetes Education Program estimates that 382 million people and 25.8 million Americans have diabetes.

That means that every day - multiple times a day - over 8 per cent of people in this country have to take time out of their day to prick themselves to test their blood levels. And because the process is so uncomfortable and difficult, it's becomes hard for a lot of people to properly manage the disease.

Or, as Google project co-founders Brian Otis and Babak Parviz said in the post: "Although some people wear glucose monitors with a glucose sensor embedded under their skin, all people with diabetes must still prick their finger and test drops of blood throughout the day. It's disruptive, and it's painful. And as a result, many people with diabetes check their blood glucose less often than they should."

Physicians and medical researchers have thought about ways to measure glucose through the fluid in the eye for years, but have had trouble figuring out how best to capture and analyze those tears reliably. Some companies, such as EyeSense, have developed their own products to embed sensors in the eye to measure these levels, while other companies such as Freedom Meditech have explored measuring glucose levels through the eye by using light.

But Google, tapping Parviz's deep knowledge of biotech, has come up with this solution. Parviz - who once led the Google Glass team - and Otis were colleagues at the University of Washington before moving over to Google's department for developing "moonshot" projects, Google[x]. The company is still in the early days of the smart contact lens project, but said that it is in discussions with the Food and Drug Administration to figure out how to bring the product to market in the future.

Hall said that the potential to improve a way to monitor diabetes is exciting, but still noted that Google's security is not the only system that users have to worry about. If it interacts with other company's apps

"One thing I do worry about is mobile security itself. It is a miasma and the app that's developed to use with this is probably going to be made by someone else," he said. "Whoever is making that app will have to answer those questions. But they haven't been answered yet because we haven't gotten that far down the line."

Monday, January 13, 2014

The Top Trends at CES 2014

The Consumer Electronics Show in Las Vegas is turning out to be a blockbuster—and with good reason. Last year's attendance topped a record 150,000, and every indication is that this year will match or exceed that number. And you would think that with Mobile World Congress looming in February that we'd see little action in the mobile space, but that's not the case.
So what's hot on the CES floor this year? It's positively littered with tablets, convertible laptops, phones, fitness gadgets, cameras, HDTVs, and cars—you name it, and you'll find a ton of it here.
CES is not only a good indicator of what's hot, but it's also indicative of what's dead. For example, Vizio made a big splash with its 2014 HDTV lineup, which lacks 3D support—a far cry from what you would have seen just a year ago. Consumers largely hated it, so now it's either buried in a submenu or gone altogether.
CES 2014 BugManufacturers aren't giving up on smartwatches yet; it was pretty clear 2013 was a trial run, and better designs are definitely in store for 2014. And while hybrid tablet and laptop designs are all the rage this year, conventional laptops and desktops are few and far between.
Even as vendors show off their latest hardware, it's still clear that software leads in overall importance. There's an increased emphasis on apps and services, and which devices are compatible with the ones that mean the most to you. A product could have a killer design or stand-out feature, but if it doesn't work with the apps and services you use, it's a non-starter.
It turns out there's still plenty of room for innovation. Basically, if you think we've already seen everything, hold onto your seat—and perhaps your credit card. From car tech to wearables, we give you the full rundown on the hottest trends at this year's Consumer Electronics Show. Most of this stuff isn't available this very moment, but it's going to be soon, so get ready.
Connected Cars
One of the biggest stories at the show this year is the connected car—nine automakers have shown up to showcase their latest tech, which is a record for CES. Considering that the North American International Auto Show takes place the very next week in Detroit, it signals an increasing emphasis on in-car technologies. The offerings are all over the map, with laser headlights on the Audi Sport Quattro Laserlight concept (pictured), 4G LTE-equipped cars, performance action cameras, and pumped-up infotainment systems are just some of the trends we're seeing. There was plenty of talk about in-car smartphone integration, particularly from Google, but now we're seeing more capabilities being built directly into the cars themselves. Another running theme: Over-the-air or otherwise modular updates  that let you upgrade your infotainment capabilities during the entire time you own the car, and as new apps and services hit the market.

Wearable Tech
Wearables were huge at CES this year. It's a huge category—dominated by fitness gadgets, but also containing smartwatches, sports gear, augmented reality glasses, and other hard-to-categorize items with wildly varied feature sets and purposes. Pebble may have the first truly desirable smartwatch with the classy-looking Steel, while Garmin and LG are challenging FitBit and Withings with new fitness bands (Garmin Vivofit is pictured below). Epson has both a fitness band and a pair of smart glasses. And even Intel is getting in the game with its own smartwatch. Wearable tech dovetails with digital health, and is ideal for monitoring your personal wellness and achieving fitness goals—and judging by the popularity of the gadgets already on the market, look for this category to explode even further as 2014 rolls on.


4K Falls to Earth, Gets (Almost) Real
The move to Ultra HD video is in full swing, with many more 4K HDTVs at far more reasonable prices than last year's five-digit early adopter sets. (For most consumers, they're still not cheap enough though, although Vizio's tempting $1,000 50-inch 4K TV certainly comes close.) We're also seeing a few prosumer-level 4K camcorders. So why say "almost" real? There's still a giant missing piece of the 4K puzzle: content. It's not like 3D, where there was virtually no good content to watch; there are in fact many movies already shot in 4K. But given bandwith limitations, getting that content into your home and watching it is another story, and whether it's by disc, by streaming Netflix, watching YouTube, or another method, it's still too early to tell just how that's going to play out.




The Toy Robot Revolution
Robot toys are suddenly big, now that the technology is there and prices are coming down. Products like the Orbotix Sphero 2B (pictured), the Ozobot, and Anki Drive show let you control and play with robotic gadgets wirelessly and control them from your smartphone or tablet. Enterprising developers can even program unique apps that work with the robots in new and exciting ways. Several of them are even promising support for multiplayer gaming, assuming you can afford to buy several. Put it this way: We've come a long way from the 1980s-era, cassette-based Omnibot.