\documentclass[12pt]{article}
\usepackage{psfig}
\usepackage{amsfonts, amssymb, amsmath, amscd, framed}
\usepackage{listings}

\usepackage{color}
\usepackage{textcomp}
\definecolor{listinggray}{gray}{0.9}
\definecolor{lbcolor}{rgb}{0.9,0.9,0.9}

\newtheorem{definition}{Definition}
\newtheorem{remark}{Remark}
\newtheorem{theorem}{Theorem}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{observation}{Observation}

\newenvironment{proof}{\noindent{\bf Proof:} \hspace*{1em}}{
        \hspace*{\fill} $\triangle$ }

\newcommand{\R}{{\mathbb R}}
\newcommand{\Z}{{\mathbb Z}}
\newcommand{\Q}{{\mathbb Q}}
\newcommand{\C}{{\mathbb C}}
\newcommand{\N}{{\mathbb N}}
\newcommand{\backtick}{{\texttt{\char"12}}}

\newlength{\toppush}
\setlength{\toppush}{2\headheight}
\addtolength{\toppush}{\headsep}

\newcommand{\htitle}[3]{\noindent\vspace*{-\toppush}\newline\parbox{6.5in}
{Q for smarties
\hfill #1 \newline
Pune\hfill#3\newline
\mbox{}\hrulefill\mbox{}}\vspace*{1ex}\mbox{}\newline
\begin{center}{\Large\bf #2}\end{center}}

\newcommand{\handout}[3]{\thispagestyle{empty}
 \markboth{ #2}{ #2}
 \pagestyle{myheadings}\htitle{\protect\ref{#1}}{#2}{#3}}


\setlength{\oddsidemargin}{0pt}
\setlength{\evensidemargin}{0pt}
\setlength{\textwidth}{6.5in}
\setlength{\topmargin}{0in}
\setlength{\textheight}{8.5in}

\lstset{ %http://www.tjansson.dk/?p=419
	backgroundcolor=\color{lbcolor},
	tabsize=4,
	rulecolor=,
	language=q,
        basicstyle=\small,
        upquote=false;
        aboveskip={1.5\baselineskip},
        columns=fixed,
        showstringspaces=false,
        extendedchars=true,
        breaklines=true,
        prebreak = \raisebox{0ex}[0ex][0ex]{\ensuremath{\hookleftarrow}},
        frame=single,
        showtabs=false,
        showspaces=false,
        showstringspaces=false,
        identifierstyle=\ttfamily,
        keywordstyle=\color[rgb]{0,0,1},
        commentstyle=\color[rgb]{0.133,0.545,0.133},
        stringstyle=\color[rgb]{0.627,0.126,0.941},
}
\begin{document}
\htitle{Professor Dennis Shasha}{q for smarties 2}{2011} 

\paragraph{Files and Interprocess Communication}

Accessing files is very easy though the syntax may look unfamiliar. 
Here for example is the code to number each line in a file.

\begin{lstlisting}
/  Numbers the lines in a file
/ q number  filename


/ APPLICATION

/ Does the numbering
myparse:{[line]
  linenum+: 1;
  :(string linenum), (": "),(line)}

/ EXECUTION

if[0 = count .z.x; til -3] / should have a file argument
file: .z.x[0]

linenum: 0 / global variable
a: read0 ` $ (":"),file / read the file

(` $ (":"),file,("out")) 0: myparse each a / parse and output the file


\end{lstlisting}
q number.q file

Points to understand:
\begin{enumerate}
\item
How to read in a file.
\item
How globals work.
\item
How to construct an output file name.
\item
How to do the output.
\end{enumerate}
\begin{framed}
Exercise 1. (Easy) 
Read in a file and count the number of characters per line then output
the numbers in the form
1: 23
2: 0
...
\begin{lstlisting}
	q countchar.q <filename>
\end{lstlisting}
\end{framed}
Now we might want to parse all the words in the file (where
words are  delimited by blanks) and count the number of instances
of words.\\
Let's work up to this.\\
We will have the same basic structure in which we read in one line
at a time and we parse it.
Only this time, parsing it will consist of separating the words
based on spaces.\\

Consider how to parse a line so we can get just the words:
\begin{lstlisting}
/ deletes all blanks
delblanks:{[x]
 ii: where not x = " ";
 x[ii] }

/ separate into words
sepwords:{[x]
  y: (" ")vs x;
  c: count each y;
  ii: where not c = 0;
  y[ii]}

\end{lstlisting}
Points to notice:
\begin{enumerate}
\item
Use of $vs$
\item
Use of $count$
\item
Need for 0 to eliminate blanks
\end{enumerate}

\begin{framed}
Exercise 2: (Medium)
Parse all the words in the file (where
words are  delimited by blanks) and find all
distinct words.
\end{framed}

\begin{framed}
Exercise 2a: (Medium)
Parse all the words in the file (where
words are  delimited by blanks) and 
count the number of instances
of words.
Display the pair (word, number) for all words having at least 3 instances
in descending order by count.
\begin{lstlisting}
	q wordcount.q filename
\end{lstlisting}
\end{framed}
In the following exercise, we read a q program and try to find
elementary syntax errors.

A q program needs semi-colons within its procedures except 
on the last line of a procedure or if we're inside parentheses.
Note that we don't want to put the semi-colon after the 
comment in a line or on blank lines.\\

\vspace{40mm}
Hints:
\begin{lstlisting}
 line: "5 + 2" / this is an addition
 (line til 6),("; "),(6 _ line)
 jj: line ss " / "
\end{lstlisting}

\begin{framed}
Exercise 3 (Middle). Read 
in a file and check that it has semicolons where needed
and add them.
\begin{lstlisting}
	q findmissing.q test.q -- Adds in semi-colons where needed
\end{lstlisting}
\end{framed}
\paragraph{Client-server communication}

Doing client-server communication is easy.\\
Let's think about the information that is required.\\
The server has to establish an address.\\
The client sends to that address.\\
The client has to make a few decisions:
\begin{enumerate}
\item 
which function to call on the server
\item
whether the call is
\begin{enumerate}
\item
synchronous (client waits for the response before proceeding)
\item
or asynchronous (client goes on).
\end{enumerate}
\end{enumerate}
For example, here is a simple server:

\begin{lstlisting}
/ q simpleserver.q -p 1111

/ Returns twice the number sent in
simple:{[num]
  2*num}
\end{lstlisting}

And here is a simple synchronous client:

\begin{lstlisting}
h: hopen`::1111; / opens the port on the current machine
h"simple[9876.5]" / calls the function with an argument
\end{lstlisting}

Here is an asynchronous client:

\begin{lstlisting}
h: hopen`::1111;
neg[h]"simple[9876.5]"
\end{lstlisting}

The trouble with the asynchronous client is that it doesn't
return anything. So we will need to keep state in the server
and call for it using a synchronous call.
Let's see if you can figure that out.

\begin{framed}
Exercise 4. (Easy) Communicate from a client to a server where the server computes
the present value of an income stream
given as a list given a certain interest rate value.
Use synchronous  calls.

\begin{lstlisting}
	q presvalserver.q -p 1234
	q presvalclient.q
\end{lstlisting}
\end{framed}

\begin{framed}
Exercise 4 async. (Easy) Communicate from a client to a server where the server computes
the present value given a certain interest rate value for four different
asynchronous calls to presvalue 
e.g.
\begin{lstlisting}
neg[h]"presval[6.5; 100 200 100 300]"
neg[h]"presval[6.0; 100 200 100 300]"
neg[h]"presval[5.5; 100 200 100 300]"
neg[h]"presval[5.0; 100 200 100 300]"
\end{lstlisting}
The client then makes a synchronous
call to collect all these results.
\begin{lstlisting}
	q presvalserverasync.q -p 1234
	q presvalclientasync.q 
\end{lstlisting}
\end{framed}

Communicating to many servers is just a matter of declaring
those servers and then using the appropriate handles.
Note you can do e.g.
\begin{lstlisting}
\sleep 3
\end{lstlisting}
if you think the server will take time.
\begin{lstlisting}
	q presvalclientasyncwithdelay.q 
\end{lstlisting}

\begin{framed}
Exercise 5. (Middle) Communicate from a client to three servers to compute the
present value assuming a variety of interest rates.
\begin{lstlisting}
	q presvalserver.q -p 1000
	q presvalserver.q -p 1001
	q presvalserver.q -p 1002
	q presvalclientmulti.q
\end{lstlisting}
\end{framed}

One way in which we might wish to parallelize jobs is to have
a reliable client which has a queue of jobs (jobqueue)
and acts as client
to a bunch of server.\\
Each time the client dispatches a job to a server it puts
the job on the back of the jobqueue.\\
Periodically, the client queries each server to see which 
jobs the server has done.\\
The client reports these in its answer queue.\\
The client eliminates all jobs that
are in its answer queue from its jobqueue.\\

pseudo-code of client:

\begin{lstlisting}
do several asynchronous calls to the servers
 for jobs that haven't completed
do synchronous calls to servers to get results
 and cause the server to reset its answer buffer
\end{lstlisting}
pseudo-code of server:

\begin{lstlisting}
put the result of each call on its answer buffer.

	q presvalserverasyncslow.q -p 10001
	q presvalserverasyncslow.q -p 10002
	q presvalserverasyncslow.q -p 10003
	q presvalclientasyncslow.q

\end{lstlisting}

\paragraph{Catching Errors}
Suppose we have a server function that performs a certain function
presval but a client calls another function foobar.
How do we prevent the client from crashing.
\begin{lstlisting}
  q presvalserver.q -p 1234
  q carefulclient.q
\end{lstlisting}
%\begin{thebibliography}{99}
%\bibitem{kxref} Kx Systems, {\it https://code.kx.com/trac/wiki/Reference}, 2011
%\end{thebibliography}
\end{document}

