Friday, 7 December 2018

Recovering disk space

If these aren't good enough:
  •  df -kh
  •  du -d 1 -h 
  •  ncdu
  •  dust

Look for deleted nodes that haven't been reclaimed yet (restart to recover space):

  •  lsof | grep '(deleted)'

Monday, 29 October 2018

Tuning PG

Our upgrade to Postgresql 10 went badly due to parallel queries. If Postgres 10 performance sucks then try turning off parallel queries.

This site generates a tuned pg config for you based on your system:

Attempt to configure your Wall-E database backs so they back up on a a time frequency not size frequency by tuning:  max_wal_size

PG Logging:

Things you really should turn on for your PG logs:


* This is useful in determining if lock waits are causing poor performance.


* Logs statements changing the DB


* Causes the duration of each completed statement to be logged if the statement ran for at least the specified number of milliseconds.  

Monday, 20 August 2018

Neat shell script functions

Greps through a file to find a single line of data containing a server name and then ssh's to it
function goto() {             
      a=$(grep -B 20 "$1" /home/andy/dev/hanson/crane/ | grep '\[' | tail -1 | cut -f 2 -d  "'")
      echo "ssh: "$a       
      ssh $a""

Sunday, 5 August 2018

*nix Directories

Root *nix Directories:


/bin - compiled ready to run programs
/dev - Devices files (disks)
/etc - System configuration files
/home - Personal directories
/lib -  Library - hold library files used by executables. (note: /usr/lib also exists)
/proc - System stats, information about running processes.
/sys - Similar to proc (TODO: Update)
/sbin - System executables. Users should NOT have /sbin in their PATH.
/usr - Not user files! File layout is similar to '/'. Historic, meant to minimise space for root user.
/var - 'Variables' Programs record runtime info. System logging, caches, user tracking etc.


/boot - kernal boot loader
/media - Removable drives
/opt - Third party software, often not used.


/usr/ - The same as above
/usr/include - Header files for C compiler
/usr/info - GNU manuals
/usr/local - For admins to install stuff here
/usr/man - man pages
/usr/share - historical from the days of low disk space

Basic Command Notes

 Refresher on Shell IO:

' > ' Is write stdout and clobber
' >> ' Is append stdout
' | ' Is stdout -> stdin
'2> ' Is write stderr
'ls badtext > out 2>&1' Redirecting stderr to the same file as stdout.

Wednesday, 28 June 2017

SQL trouble

Some SQL trouble.

This SQL doesn't do what you might think. It will update all entries in Exam_Result not just andy's exams
update Exam_Result set score = 99 FROM Student s, Exam_Result er WHERE = er.student_id and'andy'; 
This is what you wanted to do:
update Exam_Result set score = 99 FROM Student s WHERE = Exam_Result.student_id and'andy';

And this is how to fix it from a DB backup via CSV after it has gone wrong:

\copy Exam_Result (exam_id, student_id, score) to out.csv CSV;
create table fix(exam_id int, student_id int, score int);
\copy fix from out.csv CSV;
update Exam_Result set score = f.score FROM fix f where f.exam_id=Exam_Result.exam_id and f.student_id=Exam_Result.student_id;

Monday, 6 June 2016

javascript regex oddities.

Get a JS console and try this


^ This regex '.' will match the single character 'a'.

Now try with a complex unicode char like an emoji:

> ["�"]

The JS regex matches half of the unicode character. 

What is interesting is if you specify a 2 letter match JS finds the character:



In other unrelated regex bugs: \w can not understand accents:

> Null

Reading more about crazy Unicode in Javascript. Note that some accents can be displayed as letter followed by accent (2 characters) and that the same character can be letter_with_accent (1 character). Ofcourse if this happens the string length is different and they don't match.

People are upset about Python's handling of unicode too.