R by command for multiple tables

June 21, 2011 by · Leave a Comment
Filed under: Notes 

If you want to summarize a breakdown of counts by various groups in a single data frame:

x <- as.integer(runif(1000)*5)
y <- rep(c("A", "B", "C", "D"), 250)
do.call(rbind, by(x, list(y), function(g) {table(g)}))

results in:
'  0  1  2  3  4
A 54 45 45 51 55
B 52 51 56 42 49
C 60 50 49 43 48
D 51 51 51 50 47

Existence of a Solution for Matrix Equations

December 2, 2010 by · Leave a Comment
Filed under: Notes 

Thanks to this site

Let A be an m \times n matrix, x is n \times 1 and b is m \times 1

Ax = b\\ rank(A) = r\\ rank(\left[ A b \right] ) = r_a

If r_a = r < n then there are an infinite number of solutions.
If r_a = r = n then there is one unique solution.
If r < r_a then there are no solutions.

Therefore, when b is a linear combination of the columns in A, then r_a = r (since b is not independent of the other columns in A, it will not add to the rank) and there is at least one solution. But if b is not a linear combination of the columns in A then there are no solutions.

Adding a manifest to gcc-compiled .dll that links to msvcr90

November 10, 2010 by · Leave a Comment
Filed under: Notes 

First, create a manifest file, called $(DLLNAME).dll.manifest
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedPrivileges>
<requestedExecutionLevel level="asInvoker" uiAccess="false"></requestedExecutionLevel>
</requestedPrivileges>
</security>
</trustInfo>
<dependency>
<dependentAssembly>
<assemblyIdentity type="win32" name="Microsoft.VC90.CRT" version="9.0.21022.8" processorArchitecture="*" publicKeyToken="1fc8b3b9a1e18e3b"></assemblyIdentity>
</dependentAssembly>
</dependency>
</assembly>

Then make a small $(DLLNAME).rc file:

#include "winuser.h"
2 RT_MANIFEST "$(DLLNAME).dll.manifest"

Then compile this using windres in Makefile:

WINDRES = windres.exe
$(DLLNAME).o: $(DLLNAME).rc
$(WINDRES) --input $(DLLNAME).rc --output $(DLLNAME).o --output-format=coff

and link as usual:

$(BIN): $(OBJ) $(DLLNAME).o
$(CPP) $(SHARED) $(OBJ) $(DLLNAME).o $(LINKFLAGS) $(LIBS) -o $(BIN)

Thanks to this site and this one.

Extending Linux Logical Volume

November 2, 2010 by · Leave a Comment
Filed under: Notes 

Thanks to this page

First, find the volume in /dev/. On the server, this was in /dev/cciss/:

# ls
c0d0 c0d0p1 c0d0p2 c0d1 c0d1p1

c0d0 was already in the Logical Volume; we want to add c0d1.
# pvcreate /dev/cciss/c0d1p1
Physical volume "/dev/cciss/c0d1p1" successfully created

Then
vgextend VolGroup00 /dev/cciss/c0d1p1
Volume group "VolGroup00" successfully extended

Need to have installed system-config-lvm. This is a graphical tool where you can go into the Logical View of VolGroup00, “Edit Properties”, and reset the size (“Use remaining”). Click “OK” and wait for the volume to get bigger.

R Optimization

October 26, 2010 by · Leave a Comment
Filed under: Notes 

Check eigenvalues of hessian of optimized sum of squares to check for singular gradient matrix.  A singular gradient matrix has infinite solutions, so the best you can do is the optimized set of values plus a linearly scaled vector.  The vector is equal to the eigenvector associated with the eigenvalue that is numerically indistinguishable from zero.

So if you have

optimfunc <- function(x, data, sevcalcfunc, optimgoal,  ...) {
calcsev <- sevcalcfunc(x, data, ...)
sumsqerr <- sum((optimgoal - calcsev)^2)
return(sumsqerr)
}

and some function, then you can optimize
system.time(optimsimple <- optim(c(.3, -12614.716, .1), optimfunc, hessian = TRUE,data = reg.data, sevcalcfunc = calclosssimple, optimgoal = reg.data$maxloss))

and
eigen(optimsimple$hessian)
$values
[1] 1.498617e+16 1.868481e+14 8.412726e+04

Since the 3rd eigenvalue is very small compared to the first two, adding any constant z times the 3rd eigenvector to the optimized solution doesn't really change the value of the optimized sum of squares, and can therefore be considered a solution as well.

eigen(optimsimple$hessian)$vectors[,3]
[1] 1.956755e-06 1.000000e+00 4.573782e-06

> optimfunc(optimsimple$par, reg.data, calclosssimple, reg.data$maxloss)
[1] 3.793891e+14
> optimfunc(optimsimple$par + 1000 * eigen(optimsimple$hessian)$vectors[,3], reg.data, calclosssimple, reg.data$maxloss)
[1] 3.794212e+14
> optimfunc(optimsimple$par - 1000 * eigen(optimsimple$hessian)$vectors[,3], reg.data, calclosssimple, reg.data$maxloss)
[1] 3.794211e+14

Search and replace in multiple files

July 23, 2010 by · Leave a Comment
Filed under: Notes 

Thanks to this site

for fl in *.txt; do
mv $fl $fl.old
sed ’s/FINDSTRING/REPLACESTRING/g’ $fl.old > $fl
#rm -f $fl.old
done

Clear Memory Cache

June 28, 2010 by · Leave a Comment
Filed under: Notes 

Clear memory cache:

sync; echo 3 > /proc/sys/vm/drop_caches

Thanks to this site

Compile R with ATLAS

June 17, 2010 by · Leave a Comment
Filed under: Notes 

First, install yum-utils so you can run yum-builddeps:

yum install yum-utils

Then build all R dependencies with:
yum-builddep R

Add EPEL to your code repositories list, yum install atlas and wget the latest R code
rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm
yum install atlas-devel.x86_64
wget http://cran.r-project.org/src/base/R-2/R-2.11.1.tar.gz

Gunzip and tar into /usr/local/lib64, then compile R from inside R-version/

./configure --with-blas="-L/usr/lib64/atlas -lptf77blas -lpthread -latlas" --enable-R-shlib --enable-BLAS-shlib
make
make check
make install

And test:
set.seed(10)
x <- matrix(runif(1000000), 1000,1000)
y <- matrix(runif(1000000), 1000,1000)
system.time(z <- x %*% y)
#base
# user system elapsed
# 4.036 0.008 4.039
#atlas
# user system elapsed
# 0.428 0.020 0.457
set.seed(10)
x <- matrix(runif(10000000), 1000,10000)
y <- matrix(runif(10000000), 10000,1000)
system.time(z <- x %*% y)
#base
# user system elapsed
# 39.302 0.016 39.309
#atlas
# user system elapsed
# 2.840 0.028 2.873

Emacs Add New Window (for multiple displays)

June 3, 2010 by · 2 Comments
Filed under: Notes 

C-x 5 2
Ceate a new frame (make-frame-command).

R match

June 2, 2010 by · Leave a Comment
Filed under: Notes 

> a <- 1:10 > b <- 4:6 > b[match(a, b)]
[1] NA NA NA 4 5 6 NA NA NA NA

Next Page »