Background. Biases. Restart.

Have been away from blogging and working with short form social media. Namely Twitter... a forum and format has its pros and cons. Like Twitter immediacy and like the wide swath it seems to cast... at least for tech and economic news and tidbits. There is little bandwidth to indicate "who you are" on Twitter. Background often is made evident by the stream of one's posts... how happy or sad.. how positive or negative... where one's biases stand.

Privacy and humility would generally keep from disclosing.... but that is what it is to be in the public forum. It is about vulnerability and disclosure. So here is some context:

Believe the world is a generally positive place on a trajectory for better and good.

Grew up in Nova Scotia. Scottish and English ancestry. Fair share of enlisted Canadian military service in my family, though it bypassed my father and grandfather.... which is in some part why I am here. We are all the progeny of survivors - whether by luck or skill - and that biases our world view - something to think about. My immediate forbearers were farmers, foresters, millwrights and heavy equipment operators (construction trucks, tractors and bulldozers).

Grew up living in a trailer, and partially on the move, though my parents stayed put for me to stay at a consistent grade and high school. My family gave me much in the way of practical and moral compass and grounding... golden rule and trade and farm/forest skills. Learned to handle and repair farm and forest gear including firearms.  I was loved, nurtured and had a happy childhood. Which is not to say we had everything. We were sometimes hungry. We generally bought second hand. We were always on a tight budget. But living off the land, and the goodwill of family and community, always got us through.

We were cash poor but grace rich. Though we had our own sorts of family tragedies... including addiction and untimely death. But let that stand as a weak reflection to the lives of refugees, orphans and genocide survivors... We ALL live in a heaven compared to Seita and Setsuko in the Grave of the Fireflies. Have little patience with entitled attitudes otherwise.

Have worked as part of a crew in farm fields - hay, berries, husbandry, repair... hard hot/freezing honest work that gives one calluses.

Blessed in school (whether knack or persistence). But was only able to get through college because of scholarships and grants, and the goodwill and structure of the Canadian secondary education system. Was able to go to graduate school in Physics at Cornell through NSERC (Canada) funding.

Lived in a world, at least seemingly, largely devoid of criminality and corruption, and blessedly little bureaucracy. Will gladly pay taxes and fees to cover services, just do not make me do the paperwork to figure out what they will be. 

Not the first in my family through graduate education. Uncle Donald Putnam. My cousin Andrew was an MD.

Thankful for many friends through out the years. Though always sad when they drift away. But this is often how it must be. Change is the only constant. Thankful for past and ongoing opportunities to learn. Learning is my thing.... like my son Charlie, and my daughter Anna... no wait.. "purple" is her thing *grin*. And thankful for my family.

Try hard not to blame. Try hard not to remember that not everything is someone's fault. Some things are within our control. Some things are outside of our control. Sort of derivative from the Serenity Prayer.  And with that... back on to what has been going on lately...

Data Science. Feature Engineering. Sustainable Value. Seeking.

Had some exposure to data science long ago in undergraduate and graduate school:
Mapping spatial spin dynamics in helium fluids with NMR, studying positron annihilation in CuTi alloys, and coming up with good deposition recipes for making smooth gold substrates (as flat backgrounds against which to do scanning tunnelling microscopy of biologics and large chain molecules) were three that come immediately to mind.

Got some sense of theoretical underpinnings of data science from mentors at the math department at Dalhousie University. See that Dalhouse now has a data science department and will be hosting an international conference KDD-2017.  Physics Department engendered a fondness for a hands on approach. Got exposure to a wide variety of data and modeling techniques at the Condensed Matter Physics department at Cornell. Was away from data science for a long time on pathways of sensors and instrumentation as things in and of themselves. And business issues around production, logistics and customer support (of what amounts to instrumentation software). But analytics of large streams of data (sensors and tags in the Internet of Things) has brought me back to data science.

Have been taking the MIT edX course on data science. Looked at many things that this pointed to. Have become a fan of the materials at kdnuggets. And have been thinking hard about feature engineering. Note the feature engineering article is a very lean in Wikipedia, which belies its importance.

The Machine Learning Mastery article on discovering feature engineering by Jason Brownlee referenced  is well worth reading (well written and math does not overwhelm).

There are many articles about how one can excel at data science (and win Kaggle competitions *grin*) by using some core principles of collecting and understanding the data, feature engineering, applying standard or non-standard data science techniques, boosting (or model combinations).

There are very interesting new companies and business models which leverage the application of data science to seek "treasure". Some have new an wonderful techniques (like Ayasdi) and many have great tools to make a data scientist's life easier like data base stuff from Deep Information Sciences, and automatic model selection and combination like in Azure ML and IBM offers.

Cannot help but think that this all only gets one so far. The value of a model has to be harvested by deploying that model into the real world with real world constituents, and few have ventured there (or have simply taken it for granted). Further a good model often begs more data.

Those are things we have thought hard about in Analytika. One can construct a cycle:

• A. get and understand data
• B. feature engineer
• C. model and transcend
• D. deploy (get ongoing data, get ongoing analytics results)
• E. harvest business value
• F. goto A.

Thinking about how anywhere along the line we might realize a new feature in existing data or ask for....
"Can we get a new temperature sensor?" "Do you have data on turbidity?" "Do you have a flow sensor on x". State of the art is a long way from a computer or AI asking, "Do you have any data on how bright the clothing was for the Titanic survivor? How tall was each passenger?". Human inquiry and framing seems sustainably key.