Xman Blog Inside

Monday, September 18, 2006

BLOG Migration

I just got my domain name recently at http://myxman.org. That gives more much more control to post casual blog and technical blog. Hence, I shall start migrating everything from Xanga and Blogspot to this new site. I can even provide emails like xman@myxman.org, joleck@myxman.org, ivan@myxman.org Or provides space for photo postings. Currently I have only 40GB of space. I still can put in lots of stuff.

Saturday, September 16, 2006

Reversi Benchmark -- Round 2

Finally we got a Woodcrest machine for benchmark. Thanks to CopperBlue again for contributing the machine.


The Intel Xeon Dual-Core 2.66GHz (Woodcrest) achieves much better 32bit performance than AMD64X2 4200+ (2.2GHz). Running Woodcrest in 64bit achieves even higher performance.


Can you believe Woodcrest is even slower than the aging AMD64 X2? I suggest CopperBlue upgrade his Java JVM to 1.5.x. There is significant performance boost in Java 1.5 over Java 1.4. Otherwise, his latest processor is just as slow as my 1+ year-old processor. Do you care?

Labels: ,

Thursday, September 14, 2006

Reversi Benchmark: AMD64 X2 vs INTEL CORE DUO vs OTHERS

This round we manage to get an Intel Core Duo 2.16GHz with Shared 2MB L2, Mac Book Pro, to benchmark. Thanks CopperBlue for contributing his laptop. How does it compare to AMD64 X2 4200+ (2.2GHz) with 2 x 512KB Private L2 which has almost the same clock speed? However, AMD64 X2 is able to run in both 32bit mode and 64bit mode, whereas Core Duo runs only 32bit mode. Does it matter?

Let's see the first benchmark xversi-c-benchmark-v0.1 which utilizes only one core.


Intel Core Duo 2.16GHz has slight advantage over AMD64 X2 4200+ in 32bit mode. Interestingly, AMD64 X2 running in 64bit mode boost up the performance almost by 2x. Itanium II with much lower clock speed is clearly out of the game.

What about the Parallel Reversi? Let's see the next benchmark xversi-java-parallel-benchmark-v0.1.


AMD64 X2 4200+ clearly has an advantage running in 64bit mode. The 64bit JVM definitely helps in boosting the performance. Hence, if you own a 64bit processor, use a 64bit OS, and 64bit components such as Sun Java for x86_64.

Labels: ,

Monday, September 11, 2006

Python Core Language Quick Reference

If you can learn a language by example, this is a place for you to learn the core of python programming. However, some statements are mis-indented. But indentation should not be a problem to you if you are familiar with structured programming.

# Getting started.
>>> print "hello world"
hello world

xman@sai python $ cat hello.py
#!/usr/bin/python
# hello world comment
print "hello world"

>>> execfile("hello.py")
hello world

>>> import sys
>>> sys.exit()
xman@sai python $

>>> while x <= y:
... print x,y
... x=x+1
>>> x = 0 ; y = 1 ; z = 2
>>> print "%4d %0.2f" % (x,y)
100 5.00

# Conditional.
>>> if x < z =" x"> y:
... z = y
... elif x == y:
... pass
... else:
... raise RuntimeError, "Unknown"
...

# Keywords: and assert break class continue def del elif else except exec finally for from global
if import in is lambda not or pass print raise return try while
# Operators: + - * ** / % << >> < > <= >= == != <> & | ^ ~ not and or

# File.
>>> f = open("hello.py")
>>> line = f.readline()
>>> while line:
... print line
... line = f.readline()
...
>>> f.close()

# File methods
f.read([n])
f.readline()
f.readlines()
f.write(s) # Write string s.
f.writelines(l) # Write all strings in list l.
f.close()
f.tell()
f.seek(offset [,where])
f.isatty()
f.flush()
f.truncate([size])
f.fileno()
f.readinto(buffer, nbytes)

# String & Char
>>> a = "hello world"
>>> print a[4]
o
>>> a = 'hello world'
>>> b = "hello world"
>>> c = """hello world"""
>>> d = """hello
... world
... """

>>> print d
hello
world

# Slice
>>> x = a[0:5]
>>> y = a[1:]
>>> z = a[:4]

# String
>>> a = "hello" ; b = "world"
>>> c = a + b ; print c
helloworld

# List
>>> data = ["a", "b", "c"]
>>> data.append("d")
>>> data = data + ['e','f']

# Reference count
>>> a = [1,3,5]
>>> b = a
>>> a, b
([1, 3, 5], [1, 3, 5])
>>> b[1] = 100
>>> a, b
([1, 100, 5], [1, 100, 5])

# Nested list.
>>> data = ['a' , ['b','c' ]]
>>> print data[1][1]
c

# Map.
>>> import string
>>> import sys
>>> f = open("data.dat")
>>> svalues = f.readlines()
>>> f.close()
>>> fvalues = map(string.atof, svalues)
>>> print svalues
['0.1\n', '0.4\n', '0.2\n', '0.3\n', '0.24\n']
>>> print fvalues
[0.10000000000000001, 0.40000000000000002, 0.20000000000000001, 0.29999999999999999, 0.2399999999
9999999]
>>> print min(fvalues)
0.1
>>> print max(fvalues)
0.4

# Tuple (Create once, no modifications allowed)
>>> a = (1,3)
>>> b = (2,4)
>>> c = (4,)
>>> print a,b,c
(1, 3) (2, 4) (4,)
>>> print a[0], a[1]

# Range.
>>> print range(3)
[0, 1, 2]
>>> print range(1,10)
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> print range(1,10,2)
[1, 3, 5, 7, 9]
>>> for i in range(1,10,2):
... print i
...
1
3
5
7
9
a = xrange(0,10000000) # Recompute to save memory.

# Dictionary (associative array).
>>> a = {
... "a" : "va" ,
... "b" : "vb"
... }
>>> print a["a"], a["b"]
>>> a.keys()
>>> a.has_key("a")
>>> del a["a"]

# Function. Local scope by default.
>>> def swap(a,b,c=0):
... "SWAP: Documentation string"
... global msg
... if c != 0 : print msg
... return (b,a)
...
>>> a,b=swap(a,b)
>>> a,b=swap(a,b,1)

# Variable-length argument, keyword argument.
# args: Additional variables in a tuple.
# kw: Additional keyword arguments in a table.
>>> def func(x, *args, **kw):

# Class.
# def: Define a method.
# First argument of a method: the object.
>>> class stack:
... "Stack data structure"
... name = "Stack"
... def __init__(self):
... self.stack = []
... def push(self,object):
... "Append object to the end"
... self.stack.append(object)
... def pop(self):
... "Remove an object from the end and return it"
... return self.stack.pop()
... def length(self):
... "Number of objects in stack"
... return len(self.stack)
>>> s = stack()
>>> s.push("data1")
>>> s.push("data2")
>>> x = s.pop()

# Inheritance.
>>> class C(A,B):
# Encapsulation.
>>> class C:
... __x = 500

# Class relationship
>>> isinstance(c,C)
>>> issubclass(C,A)


# Exception.
>>> try:
... f = open("data.txt")
... except IOError, e:
... print "My error:" , e
...
My error: [Errno 2] No such file or directory: 'data.txt'

# Module.
# Import the module hello.py.
>>> import hello
>>> import string
>>> dir(string) # List contents of module string.

# Syntax.
# Multi-line.
>>> c = a + \
... b

# Consistent indentation.
# Codes in the same block must uses the same indentation.

# Documentation strings.

>>> def func(n):
... "Documentation string"
... return n

# Type & ID
>>> type(a)

>>> id(a)
5280552
>>> a is b
>>> a is not b
>>> type(a) == type(b)

# Copy.
>>> import copy
>>> b = [1,2,3]
>>> a = copy.deepcopy(b)

# List:
list(s)
s.append(x)
s.extend(x)
s.count(x)
s.index(x)
s.insert(i,x)
s.pop([i])
s.remove(x)
s.reverse()
s.sort([cmpfunc])

# Mapping types:
len(m)
m[k]
del m[k]
m.clear()
m.copy()
m.has_key(k)
m.items()
m.keys()
m.update(b)
m.values()
m.get(k,[,f]) # Return m[k] if found; otherwise return f.

# Unbound method object, bound method object.
# C is a class, c is an instance of class C.
>>> c.work()
hello world
>>> C.work(c)
hello world

# Functional programming
>>> a = lambda x,y : x+y
>>> print a(2,5)

# map(), reduce(), filter().
# Apply func to elements in s.
>>> t = map(func,s)
# Reduce a using sum function.
>>> b = reduce(sum, a)
# Filter objects which satisfy the conditions.
>>> b = filter(lambda x: x < style="font-weight: bold;"># Execution.
# eval(str, [,globals [,locals]])
>>> c = eval('a+b')
>>> exec "print 4+3"
# execfile(filename [,globals [,locals]])
>>> execfile("hello.py")
# compile(str,filename,kind)
>>> c = compile(str, '', 'single')
>>> c = compile(str, '', 'exec')
>>> c = compile(str, '', 'eval')

# Input, Output
>>> s = raw_input("type something")
>>> c = sys.stdin.read(1)
>>> sys.stdout = open("output.txt", "w")

# Serialize
>>> f = open("save.dat", "w")
>>> pickle.dump(a,f)
>>> pickle.dump(b,f)
>>> f.close()
>>> f = open("save.dat", "r")
>>> obj = pickle.load(f)
>>> obj
2

# Built-in Functions
abs(x)
apply(func [,args [,keywords]])
buffer(object [,offset [,size]])
callable(obj)
chr(i)
cmp(x,y)
coerce(x,y)
compile(string, filename, kind)
complex(real [,img])
delattr(obj, attr)
dir([obj])
divmod(a,b)
eval(expr [, globals [,locals]])
execfile(filename [,globals [,locals]])
filter(function, list)
float(x)
getattr(obj,name)
globals()
hasattr(obj,name)
hash(obj)
hex(x)
id(obj)
input([prompt])
intern(string)
isinstance(obj,class)
issubclass(subclass,class)
len(s)
list(s)
locals()
long(x)
map(function, list, ...)
max(s [,args, ...])
min(s [,args, ...])
oct(x)
open(fname [, mode [, bufsize]])
ord(c)
power(x, y, [,z])
range([start,] stop [,step])
raw_input([prompt])
reduce(func, seq [,initializer])
reload(module)
repr(obj)
round(x [,n])
setattr(obj, name, value)
slice([start,] stop [,step])
str(obj)
tuple(s)
type(obj)
vars([obj])
xrange([start,] stop [, step])

# Built-in Exceptions
Exception
StandardError
ArithmeticError
LookupError
EnvironmentError
AssertionError
AttributeError
EOFError
FloatingPointError
IOError
ImportError
IndexError
KeyError
KeyboardInterrupt
MemoryError
NameError
NotImplementedError
OSError
OverflowError
RuntimeError
SyntaxError
SystemError
SystemExit
TypeError
ValueError
ZeroDivisionError

# Internal.
>>> a = [1,4,6]
>>> a.__doc__
"list() -> new list\nlist(sequence) -> new list initialized from sequence's items"
>>> class C:
... a = 0
... def work(self):
... print("hello world")
...
>>> c = C()
>>> c.__dict__
{}
>>> C.__dict__
{'a': 0, '__module__': '__main__', 'work': , '__doc__': None}
# There are many other built-in attributes for classes, instances, modules ...

Reference: Python Essential Reference by David M. Beazley

Labels: ,

Saturday, September 09, 2006

Setup Passwordless SSH

It's convenient to setup passwordless SSH in a closed environment. Connecting to remote hosts becomes handy. Passwordless SSH Setup Step-by-Step:
1. Generate RSA public and private keys.
xman@sai ~ $ ssh-keygen -t ssh-rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/xman/.ssh/id_rsa):
Created directory '/home/xman/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/xman/.ssh/id_rsa.
Your public key has been saved in /home/xman/.ssh/id_rsa.pub.
The key fingerprint is:
xx:xx:xx:xxcommand prompt PS for root:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx xman@sai

You have to enter empty passphrase, trading off security for the convenient connecting to remote hosts.

2. Append the content of the generated public key, id_rsa.pub, into the file $HOME/.ssh/authorized_keys at the remote host. The folder .ssh and the file authorized_keys at remote host must not allow group, or other access, in particular, set the permissions of .ssh to 700 and authorized_keys to 600.

Now you can simply "ssh remotehost" without the need to type password. :) The first time you connect to remote host using ssh:
xman@sai ~ $ ssh remotehost
The authenticity of host 'remotehost (192.168.1.100)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'remotehost' (RSA) to the list of known hosts.
Last login: Sat Sep 9 23:50:33 2006 from localhost
xman@remotehost ~ $

Labels:

Friday, September 08, 2006

Make Quick Reference

Inside Makefile:
TARGET: DEPENDENCY LINE OR RULES LINE
COMMAND
COMMAND
...
A command line (with TAB in front) is running in a subshell by itself.
# A comment.
name = text in nameMacro definition.
$ASingle character macro reference.
$(name) or ${name}Macro references.
make -pPrint internally defined macros.
make target "DIR=/dir /dir2"Use DIR defined in command even if you defined DIR in your Makefile!
DIR=/bin make targetUse DIR in Makefile if any, otherwise use DIR=/bin.
make -eShell variables have higher priority than file macro definitions.
FILE=a.c b.c
@echo ${FILE:.c=.o}
a.o b.o
Replace suffix .c by .o. The @ suppress the printing of commands executed.
$@Current target. Use in command line only.
$?List of pre-requisites newer than current target. Use in command line only.
$$@Current target. Use in dependency line only.
$*File name without suffix part. Use in commands in suffix rule only.
$%Name of the .o when target is a library module.
$<The pre-requisite file. Use in commands in suffix rule only.
.SUFFIXES: .o .cSuffixes that make will consider.
.c.o:
${CC} -c $<
Suffix rule. To produce .o from a .c.
t: libm.a(sin.o)Target t depends on sin.o in libm.a. Hence, if sin.o is newer than libm.a, implicit rule replace newer sin.o into libm.a, then create target t.
libm.a :: sin.c
libm.a :: cos.c
Rebuild libm.a using different commands depends on the pre-requisites.
- rm -f *.oIgnore errors in the command rm, hence, errors will not stop the make process.
t: a.c \
b.c
Use '\' to continue with the next line.
.SUFFIXES:
.SUFFIXES: .a .b
.SUFFIXES: .c
1. Clear implicit suffixes. 2. Define .a .b suffixes. 3. Add .c into the suffix list.
.c.a:
true
Override default suffix rule with null command
t:
echo $$HOME
Use $$ in commands for environment variable.
.IGNOREForce make to ignore errors and keep going.
${<D}, ${<F}Pre-requisite file directory, pre-requisite file name.
VPATHView path.
include inc.mkInclude the file inc.mk.
.PRECIOUS: a.x b.xDo not let make deletes files a.x and b.x.
.SILENTDo not echo commands while executing.
.DEFAULT:
echo "hello"
Define a default action.

Reference: C Programming Utility: Managing Projects with make by Andrew Oram and Steve Talbott

Labels: ,

Wednesday, September 06, 2006

Gawk Quick Reference

By default, a record is a line. A line is made up of fields with default delimiter. Main programs are mainly based on C program syntax.
BEGIN { FS="\t+" }Initialization. Set field separator to be one or more TABs. Take the string as regular expression if more than 1 characters.
END { ... }Finalization. Execute ... after all files have been processed.
gawk -F"\t" '{ ... }'Execute with field separator TAB.
/xyz/ { print }For each line contain xyz, print the line.
$1 == 100 { print $2, $3 }For each line with first field equals 100, print the 2nd and 3rd fields separated by space.
$3 ~ /PAT/ { print $2 $3 }If the 3rd field matches PAT, print the concatenated 2nd and 3rd fields.
$3 !~ /PAT/ { x = 0 }If the 3rd field doesnt match PAT, let x = 0.
print $1 OFS $2Print 1st and 2nd fields separated by output field separator.
FS, OFS, RS, ORSField & output field separator, record & output record separator.
NF, NRNumber of fields and records.
FILENAMECurrent input file.
$NFLast field
'${val:-hello}'In matching region, represents value of $val from the BASH environment, but use "hello" if val is not defined.
"'${val}'"In code region, represents value of $val from the BASH environment.
array[2]="hello"
array["i"]="world"
for(i in array) print array[i]Also works for multi-dimensional array.
if("i" in array) print "found"Print if "i" is a subscript of array.
n = split($1, array, ":")
for(i = 1; i <= n ; i++) print array[i]
Split 1st field into array with ":" as delimiter.
array[2,5]="val25"Equivalent to array["2" SUBSEP "5"]="val25". SUBSEP is a subscript-component separator such as "\034" by default.
if((i,j) in array)
ARGC, ARGV, ENVIRONNumber of arguments, argument array, and environment array.
index(s,t)Position of t in s.
length(s)
sub(r,s[,t])Substitute first match of r in t by s. t is $0 by default.
match(s,p)Return starting position of the substring in s that regular expression p matches.
sprintf("fmt",expr)
substr(s,p,n)The substring of s at position p up to n long.
toupper(s), tolower(s)
function name(list) {
statement
}
getlineGet next line.
getline <"-"Get a line from stdin.
print > "out.txt"Print to the file out.txt.

Reference: UNIX Power Tools: sed & awk by Dale Dougherty

Labels: ,