10 minute rollup

calculate the last price and sum of size of 10 minute intervals of trade
select last price, sum size by 10 xbar time.minute from trade where sym=`MSFT

hourly mean variance of some symbol
select mean:avg price, variance:var price, stdDev:dev price by date, hour:time.hh from trade where sym=`AIG

Average total trade volume over a certain week, for each 30 minute bucket for MSFT
(select sum size by 30 xbar time.minute from trade where date within 2006.08.07 2006.08.11,sym=`MSFT)% sum date within 2006.08.07 2006.08.11

Select the two heaviest traded stocks
select sym from 2# desc select sum size by sym from trade

This dyadic function shifts a vector by n elements, for example
select price-5 xprev price from trade where sym=`IBM

Generate a NBBO table for a certain month: (National best bid and offer)
nbbo: select ask: min ask,bid: max bid by date,sym,time from quote where date.month=2006.08m,asize>0,bsize>0

non-first normal form can be faster
q) \t do[10000;r1:select avg price by sym from trade where sym in `AIG`IBM] 1862
compared to
q) \t nestedtrade:select price,size by sym from trade 0 q) \t do[10000;r2:select sym,each[avg] price from nestedtrade where sym in `AIG`IBM] 180

=======

Set up a console

q trade.q -p 5001

http://localhost:5001
http://localhost:5001/?select from trade where sym=`IBM

export to excel:

http://localhost:5001/.csv?select from trade where sym=`IBM

\a set of tables
\c 20 200 how much to display
\v all global variable
\date -- operating system date ommand

========

Interlanguage access -- do excel, web, and maybe java
p. 112 of fdkdbtutorial

// build (time;sym;price;size) record
Object[]x={new Time(System.currentTimeMillis()%86400000),"xx",new
Double(93.5),new Integer(300)};
c.k("insert","trade",x);	// insert[`trade;x]
clients can also read incoming messages:
r=c.k();

for example, kdb+tick clients do:
String[]syms={"IBM","MSFT",..};
c.k(".u.sub","trade",syms);	// subscribe
while(1){Object r=c.k();..	// process incoming


p. 126 of the tutorial

import c; // That is the critical java to kdb module to import

// I don't know which of these to import
import java.awt.*; 
import java.awt.event.ActionEvent; 
import java.io.IOException;
import javax.swing.*; 
import javax.swing.table.DefaultTableModel;
// end of I don't know what

String host = "localhost";
int port = 6001;
qConn = new c(host, port); connects to a database specified by host and port

Object o = qConn.k(sSql); 
	// access the database and gets info based on sql query

c.Flip d = c.td(o); // columns are d.x and rows are d.y

int numcols = c.n(d.x); 
int numrows = c.n(d.y[0]);

Object[][] myobj = new Object[numrows][numcols];

for (int rowid = 0; rowid < numrows; rowid++) {
  for (int colid = 0; colid < numcols; colid++) {
	myobj[rowid][colid] = c.at(d.y[colid], rowid); 
		// fishes out the colid field of row rowid
  };
};



clients (see http://www.studio4kdb.com/ExcelRTDServer.htm)
From julian.dobson@baml.com  Fri Mar 15 15:09:03 2013
Received: from mx.cims.nyu.edu (mx.cims.nyu.edu [128.122.49.99])
	by mail.cims.nyu.edu (8.14.5+Sun/8.14.5) with ESMTP id r2FJ93Gf024069
	for <shasha@mail.cims.nyu.edu>; Fri, 15 Mar 2013 15:09:03 -0400 (EDT)
Received: from vadmzmailmx04.bankofamerica.com (vamx04.bankofamerica.com [171.159.192.80])
	by mx.cims.nyu.edu (8.14.4+Sun/8.14.4) with ESMTP id r2FJ8rHH006829
	for <shasha@courant.nyu.edu>; Fri, 15 Mar 2013 15:08:53 -0400 (EDT)
Received: from vadmzmailmx05.bankofamerica.com ([171.182.203.230])
	by vadmzmailmx04.bankofamerica.com (8.14.5/8.14.5) with ESMTP id r2FJ8lmm027651
	for <shasha@courant.nyu.edu>; Fri, 15 Mar 2013 19:08:48 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=baml.com; s=corp1210;
	t=1363374528; bh=Q6cjOebwQuA9GW31EPSN2DNmzQbmuUa8SzHNI7fdSfg=;
	h=Date:From:Subject:To:Message-id:MIME-version:Content-type;
	b=dCQSUUPJFRwzMQ9oCzHNo28MnvJL4+JsaYZm0SsnB4dadnNnI9W0UWNlRv/IyawK4
	 HW4XZyN2zu0yPamBm02KCSQgQlaOyiZKtkMEGLAgg8mYj5BIh0lQYed3JncWx42Rgy
	 zBcxMuiPT8V7jpe+XYkvzXHIGzHcsrxiLWTew4zM=
Received: from memva2mta01.bankofamerica.com (memva2mta01.bankofamerica.com [171.186.140.73])
	by vadmzmailmx05.bankofamerica.com (8.14.5/8.14.5) with ESMTP id r2FJ8THI000341
	for <shasha@courant.nyu.edu>; Fri, 15 Mar 2013 19:08:47 GMT
Date: Fri, 15 Mar 2013 19:08:45 +0000
From: "Dobson, Julian M - GMRT-EST" <julian.dobson@baml.com>
Subject: RE: Sorry, another q question
X-Originating-IP: [171.206.139.225]
To: Dennis Shasha <shasha@courant.nyu.edu>
Message-id: <3146837794DF29488A9F2C3CCC1783D23A19FE5B@smtp_mail.bankofamerica.com>
MIME-version: 1.0
Content-type: multipart/alternative; boundary="Boundary_(ID_a1ufm7VjQiQ2pgkus5c+Dg)"
Content-language: en-US
Accept-Language: en-US
Thread-topic: Sorry, another q question
Thread-index: Ac4hry6zYCAuYSwoTuuMYhFregAOUwAASnPQ
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.10.8626,1.0.431,0.0.0000
 definitions=2013-03-15_08:2013-03-15,2013-03-15,1970-01-01 signatures=0
X-Proofpoint-Spam-Reason: safe
X-Greylist: Sender IP whitelisted by DNSRBL, not delayed by milter-greylist-4.4.1 (mx.cims.nyu.edu [128.122.49.99]); Fri, 15 Mar 2013 15:08:54 -0400 (EDT)
X-Virus-Scanned: clamav-milter 0.97.4 at mx
X-Virus-Status: Clean
X-Scanned-By: MIMEDefang 2.73 on 128.122.49.99
Status: RO
Content-Length: 8522


--Boundary_(ID_a1ufm7VjQiQ2pgkus5c+Dg)
Content-type: text/plain; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT

Interestingly enough, if I convert tbl to a string, and use that in the value:

tbl:"flip (stags)$ flip ctags xcol { tags #.[!] \"S=\\001\" 0:x} each fx";
idx:value "exec i from (",tbl,") where ",filter;

this works fine.

Julian Dobson
GEF&S Technology
Bank of America Merrill Lynch
Merrill Lynch, Pierce, Fenner & Smith Incorporated

212 670 1720

      _____________________________________________
      From: Dobson, Julian M - GMRT-EST
      Sent: Friday, March 15, 2013 2:59 PM
      To: 'Dennis Shasha'
      Subject: Sorry, another q question


      Dennis,

      I have a function that is applying a filter to a list of FIX messages:

      set_fxmsg:{[fx]
        tbl:flip (stags)$ flip ctags xcol { tags #.[!] "S=\001" 0:x} each fx;
        idx:exec i from tbl where c74000=`HOTS,c35=`8,c150 in `0`F`G`H`4`5,(((c96051=`HOTS) and ((c5054 in `PARENT`CHILD) or ((c5054=`) and (c5035=`)))) or ((c96051=`ED) and (c5054 in `CHILD`SORT_CHILD)));
        idx2:value "exec i from tbl where ",filter;

      tbl is basically a list of the fix tags that you want to filter, so would expand to something like:

      flip ("SSSSSS")$ flip `c74000`c35`c150`c5035`c5054`c96051 xcol {`74000`35`150`5035`5054`96051#.[!] "S=\001" 0:x} each fx

      idx is a vector of indices that pass the filter, and this works fine.

      In idx2, filter is set to the where clause from idx - and this fails with:

      'tbl
      .:
      "exec i from tbl where c74000=`HOTS,c35=`8,c150 in `0`F`G`H`4`5,(((c96051=`HO..

      Once in debug, I can redo idx2 and it works fine. What am I doing wrong (the whole exercise is to move the filter into config, as in:

      tags: value inst.props[`filter.tags];
      ctags: value ssr[inst.props[`filter.tags];"`";"`c"];
      { stags,:"S"} each til count tags;
      filter: inst.props[`filter];

      Thanks.

      Julian Dobson
      GEF&S Technology
      Bank of America Merrill Lynch
      Merrill Lynch, Pierce, Fenner & Smith Incorporated

      212 670 1720


----------------------------------------------------------------------
This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.

--Boundary_(ID_a1ufm7VjQiQ2pgkus5c+Dg)
Content-type: text/html; CHARSET=US-ASCII
Content-transfer-encoding: 7BIT

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<font face="Calibri" size="2"><span style="font-size:11pt;">
<div><font color="#1F497D">Interestingly enough, if I convert tbl to a string, and use that in the value:</font></div>
<div><font color="#1F497D">&nbsp;</font></div>
<div><font color="#1F497D">tbl:&quot;flip (stags)$ flip ctags xcol { tags #.[!] \&quot;S=\\001\&quot; 0:x} each fx&quot;;</font></div>
<div><font color="#1F497D">idx:value &quot;exec i from (&quot;,tbl,&quot;) where &quot;,filter;</font></div>
<div><font color="#1F497D">&nbsp;</font></div>
<div><font color="#1F497D">this works fine.</font></div>
<div><font color="#1F497D">&nbsp;</font></div>
<div><font face="Arial" size="2" color="#1F497D"><span style="font-size:10pt;">Julian Dobson</span></font></div>
<div><font face="Arial" size="2" color="#1F497D"><span style="font-size:10pt;">GEF&amp;S Technology</span></font></div>
<div><font face="Arial" size="2" color="#1F497D"><span style="font-size:10pt;">Bank of America Merrill Lynch</span></font></div>
<div><font face="Arial" size="2" color="#1F497D"><span style="font-size:10pt;">Merrill Lynch, Pierce, Fenner &amp; Smith Incorporated</span></font></div>
<div><font face="Arial" size="2" color="#1F497D"><span style="font-size:10pt;">&nbsp;</span></font></div>
<div><font face="Arial" size="2" color="#1F497D"><span style="font-size:10pt;">212 670 1720</span></font></div>
<div><font color="#1F497D">&nbsp;</font></div>
<div style="padding-left:36pt;"><font face="Tahoma" size="2"><span style="font-size:10pt;">_____________________________________________<br>

<b>From:</b> Dobson, Julian M - GMRT-EST <br>

<b>Sent:</b> Friday, March 15, 2013 2:59 PM<br>

<b>To:</b> 'Dennis Shasha'<br>

<b>Subject:</b> Sorry, another q question</span></font></div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">Dennis,</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">I have a function that is applying a filter to a list of FIX messages:</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">set_fxmsg:{[fx]</div>
<div style="padding-left:36pt;">&nbsp; tbl:flip (stags)$ flip ctags xcol { tags #.[!] &quot;S=\001&quot; 0:x} each fx;</div>
<div style="padding-left:36pt;">&nbsp; idx:exec i from tbl where c74000=`HOTS,c35=`8,c150 in `0`F`G`H`4`5,(((c96051=`HOTS) and ((c5054 in `PARENT`CHILD) or ((c5054=`) and (c5035=`)))) or ((c96051=`ED) and (c5054 in `CHILD`SORT_CHILD)));</div>
<div style="padding-left:36pt;">&nbsp; idx2:value &quot;exec i from tbl where &quot;,filter;</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">tbl is basically a list of the fix tags that you want to filter, so would expand to something like:</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">flip (&quot;SSSSSS&quot;)$ flip `c74000`c35`c150`c5035`c5054`c96051 xcol {`74000`35`150`5035`5054`96051#.[!] &quot;S=\001&quot; 0:x} each fx</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">idx is a vector of indices that pass the filter, and this works fine.</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">In idx2, filter is set to the where clause from idx &#8211; and this fails with:</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">'tbl</div>
<div style="padding-left:36pt;">.:</div>
<div style="padding-left:36pt;">&quot;exec i from tbl where c74000=`HOTS,c35=`8,c150 in `0`F`G`H`4`5,(((c96051=`HO..</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">Once in debug, I can redo idx2 and it works fine. What am I doing wrong (the whole exercise is to move the filter into config, as in:</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">tags: value inst.props[`filter.tags];</div>
<div style="padding-left:36pt;">ctags: value ssr[inst.props[`filter.tags];&quot;`&quot;;&quot;`c&quot;];</div>
<div style="padding-left:36pt;">{ stags,:&quot;S&quot;} each til count tags;</div>
<div style="padding-left:36pt;">filter: inst.props[`filter];</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">Thanks.</div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;"><font face="Arial" size="2"><span style="font-size:10pt;">Julian Dobson</span></font></div>
<div style="padding-left:36pt;"><font face="Arial" size="2"><span style="font-size:10pt;">GEF&amp;S Technology</span></font></div>
<div style="padding-left:36pt;"><font face="Arial" size="2"><span style="font-size:10pt;">Bank of America Merrill Lynch</span></font></div>
<div style="padding-left:36pt;"><font face="Arial" size="2"><span style="font-size:10pt;">Merrill Lynch, Pierce, Fenner &amp; Smith Incorporated</span></font></div>
<div style="padding-left:36pt;"><font face="Arial" size="2"><span style="font-size:10pt;">&nbsp;</span></font></div>
<div style="padding-left:36pt;"><font face="Arial" size="2"><span style="font-size:10pt;">212 670 1720</span></font></div>
<div style="padding-left:36pt;">&nbsp;</div>
<div style="padding-left:36pt;">&nbsp;</div>
</span></font>

<HR>This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended recipient, please delete this message.<BR>
</body>
</html>

--Boundary_(ID_a1ufm7VjQiQ2pgkus5c+Dg)--

