A period of changes

For a rather obvious reason people tend to segment the time into orbital rotations of planet earth. As an earth dweller I too utilize to that convention, maybe a bit too much, but whatever. Following this tradition, I decided to write up few thoughts looking back on 2015. It’s going to be a fairly personal blog entry. My main focus is not going to be on making deductions or inductions on  common topics, but I will not refrain entirely from doing so either.

Despite being fairly content at the end of last year, I very impulsively (and with a lot of hesitation at first) moved to NYC from central Illinois (Champaign-Urbana). Initially the move was temporary, and at longest it was going to last for six months. But then I decided I wanted to stay longer, and eventually the move resulted in me getting transferred to Columbia from UIUC for the entire PhD thing.

Changing schools is by itself a big move, but it isn’t something I did not experience before. (I also transferred during undergrad and back in middle school) But this year also came with some other life style changes, that I did not really taste before. I think overall, I increased the bandwidth allocated to external communication a bit, and it resulted in tasting some social reality. I am not saying I was able to make a quantum leap in my social game, but I have been able to take a much more relaxed stance and reduce my overall ‘neediness’ and social desperation.

The relaxation always comes to me with observations (or equivalently  experience, for those who are not into machine learning lingo). More I get to collect observations from a certain aspect of life, more my self-esteem in that particular channel is boosted. E.g., I played music with a lot of people this year. I feel much more like conquering territorial beast compared to before in the music space!

Regarding my research life (my job that is) I have decided that I would like to be involved in projects that involve more brains rather than rush. I haven’t been able to live up to my standards in my own research especially in the second half of the year. I will try to mitigate my concerns in this area this year.

But overall, I guess the most interesting transformation is regarding my objective function. Up to this year I have been more or less a selfish and scared person. This year I somehow managed to build an interest for others – the world outside. The thoughts regarding betterment of humanity and human experience began to surround me, to the point where it sometimes clouded my clairvoyance and self determination. This shift is no doubt influenced by the crappy state of today’s world based on I-Me-Mine. This year I began to like the Pope for the first time. I learned that he’s not only an influential leader who preaches against capitalism and warns about the lurking dangers of global warming, but also a rocker! I also stopped eating animals this year, partly motived by environmental concerns and partly thinking for my health!

I have also been able to firmly decide that the lofty peak I am longing to reach, irrespective of whatever it is, should be reached without compromising from myself. Maybe that lofty peak is me/us, and all this is about finding who I am/we are..

Yeah, let me conclude with pictures of the rooms I slept in this year:

My place in Urbana with Jo in picture

The place in East Harlem I initially went:

My later room in NYC this spring

My room this summer

My current hole:

NIPS 2015: few things to note, thoughts, moments

It has been a good NIPS. I got my visa at the last minute (I wasn’t decisive on going since I don’t have anything to contribute this year and I can not decide fast).  I am kind of glad that I put in the effort in getting it,  it was my first NIPS without any networking pressure, so I was able to channel more thoughts more on the good stuff. Here are few papers that I will probably go back and read:

Estimating Mixture Models via Mixtures of Polynomials
Sida Wang · Arun Tejasvi Chaganty · Percy S Liang
-interesting
Automatic Variational Inference in Stan
Alp Kucukelbir · Rajesh Ranganath · Andrew Gelman · David Blei
-might make life slightly easier for me
Matrix Manifold Optimization for Gaussian Mixtures
-interested in manifold optimization in general
Sparse and Low-Rank Tensor Decomposition
Parikshit Shah · Nikhil Rao · Gongguo Tang
-I like tensors and factorizations
Mixing Time Estimation in Reversible Markov Chains from a Single Sample Path
Daniel J Hsu · Aryeh Kontorovich · Csaba Szepesvari
-I like chains of Markov, and my advisor is an author
Gradient Estimation Using Stochastic Computation Graphs
John Schulman · Nicolas Heess · Theophane Weber · Pieter Abbeel
-had a good conversation during the poster presentation
Besides technical stuff I am really glad that they had the symposium on societal impacts of machine learning. I liked the discussions in general and I am particularly happy that they talked about economic consequences  of AI. I highly recommend checking out the talk by  Erik Brynjolfsson on the subject once the talks get uploaded. The idea I got is, although the GDP saw a rise after AI, the median income person regressed few steps in the social ladder: It seems like AI is indeed helping the rich get richer. I would have liked them to talk more on this problem, but they rather allocated more time on singularity and similar catchy topics.
Here’s the symposium website by the way:
On a more personal note, I liked the week in Montreal. I have been able to contemplate on few general things about people, me and few general things. It seems like I may be able to find solace in pure curiosity. On another note, after few interactions during this week, I am more and more getting inclined to think that people are inherently ‘good’ beings and it pays off to getting to reach out to that inner ‘goodness’. – yeah sounds naive- Connecting with people is definitely a winding road with no convergence guarantee, but it no doubt deserves a try . People do seem and act far from promising – even despicably-  at times, but I guess I don’t see the point in raising the levels of hostility at this point in time. Note that I am saying all these things for a species which is on the verge of planetary suicide.
Here are few moments from the week in Montreal:
There was way too many people!
Will algorithms help these guys?
Seems like we are not alleviating the income inequality – from the algorithms among us symposium
disturbing…
even more disturbing..
not all ‘machines’ are inorganic..
this is not related to the conference except that it was in ‘palais des congres’. I like the colors.
Please do comment, suggest papers, share thoughts if you want to..
Cheers,
cem

[Discussion] It sounds good, is it good?

Here’s Johannes’ original message sent to me regarding the subject:

This is one of the more substantive on-line discussions I’ve read so far on the topic:

Interesting. Apparently, Duke Ellington said that “if it sounds good, it is good.” I disagree vehemently. I’ve had many experiences where I listen to something for the first time and get super bored thinking it’s just a bunch of notes or even just noise. Perhaps to your surprise, examples include Chopin’s 4th Ballade, Stravinsky’s Rite of Spring, all of Beethoven’s late sonatas, and Prokofiev’s piano concertos.
I remember having a discussion with a music teacher at my high school about Beethoven where I basically said “Moonlight, Pathetique, and Appassionata are the best, the later stuff is just weird.” To which he replied, “I think a lot of people, including myself, find the late sonatas to be his best work.” To which I replied, “that sounds crazy to me.” Now, I’m much more tempted to agree with him. The 4th Chopin Ballade I heard for the first time in 10th grade, roughly, and I had a conversation with a pianist friend in the grade above me who said he really liked it. But I could only say, “I don’t get it, what is this supposed to be?” And Stravinsky I heard for the first time in the summer before my last year of high school. A math + bass guy in the summer camp had me listen to it (his orchestra was working on it) and he said that he really liked it. But I just found it dry and unpleasant. And Prokofiev, of course, is often still considered “weird” even though I’ve grown very fond of some of his sonatas and piano concertos.
Nowadays, all that music is among my all-time favorites. So what I’ve learned is that for complex or unfamiliar music, you first have to come to terms with the language in which it is written. Once informed, you can make an educated decision about whether or not you enjoy it on aesthetic grounds. Duke Ellington was advocating for the “natural” approach in that quote, but unfortunately, ignorance is also natural (and limiting)
Posted in Discussion, music | 2 Comments

Ten albums

I suddenly decided to list ten albums that I classify as “stellar”. Don’t take it as a “must listen before you die” list. They are not even my “best” 10 albums. I don’t do that because that would be too predictable. (Sgt.Peppers is everybody’s favorite right?) They are just random 10 great albums to have great time with..

1. Heritage – Opeth (2011)
What a great (and dark) progressive rock album. If you are a prog rock fan, you just can’t  miss this one. Beautiful chord progressions enhanced with the angelic voice of Akarfeldt (not demonic in this case). I just can’t have enough of these sound waves emanating from my loudspeakers..
2. Raven that refused to sing – Steven Wilson (2013)
Pure poetry, progressive rock at its height with perfect story telling. A sad story of an old man, who is near the end. Beauty embedded in sadness..
3. Balance – Van Halen (1995)
A great album with catchy hits. Sammy Hagar’s voice at its height.
4. Satilik (For Sale) – Yavuz Cetin  (2001)
Yavuz Cetin, without any national bias or anything, to my mind is one of the greatest blues rock guitarists of all time. This is the lifetime statement  of a guitar hero, and a awesome human being. A farewell to the world. He said: “I do not wanna live in your society, you hypocrites”. in his masterpiece. And he sailed away as he promised. In my opinion one of the greatest blues-rock albums of our times. Pure awesomeness forged in the vast mountains of a hero’s heart..
5. Rage Against the Machine – Rage Against the Machine (1992)
This is an album purely about denying the authority, standing up for freedom and justice. I was fortunate enough to hear “Wake up”  on Acik radio 94.9 in 3am of 1st of June 2013, the first day of the public uprisings in Istanbul. What an epic moment, simultenous with the waking up of a nation against the tranny of a dark-age seeking Islamist government. To me, this album can be used as a 101 course to bombastic guitar riffs in every aspect: Guitar tone, attitude, rhythm and of course soul. Morello indeed nailed it.
6. Tokyo Jukebox 1 & 2 – Marty Friedman  (2009 & 2011)
I was initially real skeptical towards these albums. Primarily because it is labeled as J-Pop. However, it is nothing that comes near to being monotonic and stale as I expected. It’s horrendously dynamic, and full of sincere guitar playing. Surely lots of orgasmic guitar solos from the master, Marty.
7. Fuzz Universe – Paul Gilbert –  (2010)
Gilberto concerto this is! Super fun guitar playing, with ultra dynamic song structures that can keep you awake in 4 am in the morning writing up something urgent.
8. Ain’t life grand grand – Slash’s snakepit (2000)
A great album, full of energy, power, big-fat crunchy guitar riffs and of course Slash. In his prime, we hear some great guitar melodies from the living legend. His usage of high pitch passionate solos help to solidify his legendary status.
9. Too old to Rock’n Roll: Too Young to Die – Jethro Tull  (1976)
Great concept album. Very easy for a modern day man to relate himself to the protagonist in the story  🙂
10. Six Degrees of Inner Turbulence (Part Two) – Dream Theater (2002)
A great concept album about psychological disorders. Great music, great lyrics. Makes   the listener to obtain a more subtle glance towards the inner turmoils that everyone is vulnerable to, thanks to the sheer beauty of the lyrics and the music. I guess LaBrie’s vocals is one of the key elements of this album.
Posted in Uncategorized | Tagged , | 1 Comment

Several years ago, a senior engineer told me that engineering is about to be aware of the “terminology”. At that time, (two years before I started by undergraduate studies in engineering),  I could not make sense out of what he said and concluded that it was an incomplete and clumsy statement.

As we are all aware of, people give names to be able to classify things. These things are both abstract and concrete. We have the apple which triggered all and integral operator, a cab and  transistor. The thing is, these objects all imply great deal of other things. For instance, a cab imply an engine, a chassis, a driver, a client, a wheel, a door, a screw, a glass, a rim, gasoline, a radio, a loudspeaker, a mirror and so on. And if you dig further, loudspeaker implies a coil, a magnet, a … . Please note that I only listed  physical entities. We can also sort ideas related to these “names we give”. A  chassis may be connected to a Computer Aided Design tool, which made it possible to be exist as it is. In turn, a CAD tool may make us think about the programming language it is written with. Yes, we can dig until the center of the earth, but what I try to point is that more advance the things become, more do they imply or refer.

The process of learning starts first with apples and pears. We first try to understand the  “things” in their raw nature. But then, approximately ten years later, we find ourselves for example talking about evaluating a spectrogram of a human voice. The concept of spectrogram contains the very same concept of apples and pears, but also a very great deal of many other stuff.  We have time and frequency, which are distinct entities, which are not to be collectively operated, but also we have the Fourier transform and related to it we have the concept of projection, the basis vectors, the concept of complex numbers, discrete and continuous number spaces, the concept of windowing, and distortion in frequency domain, aliasing, the sampling, sampling frequency and related to it the famous Nyquist theorem…

What did I just do, was to use the terminology. Notice that, to make this spectrogram stuff, I did not break all the way down to the bottom, the lowest level. To the level, where I could not descend any more down to the ground.  Whether it is possible for a human being to represent these high-level concepts with apples and pears may be a another blog entry, but, it is certain that we do this high level abstraction to provide ease in self and inter communication. Imagine how hard it would be to define a spectrogram to first year elementary year kids. It is hard because you have make your definition in terms of apples. Pears? No we will learn pears when we grow up.

It is not hard to interpolate this result to conclude that more abstract, dense and compact the form of communication is, more we can advance. With a very low-level programming languages like assembly, indeed it is hard to implement sophisticated stuff. Even with C you may lose some hair. To me, the greatest things of my undergraduate study was my meeting with MATLAB. Suddenly, without telling too much, I was able to convey my thoughts to the stupid machine. I will never forget drawnow. It was a life change.

To conclude, I can say that, we human beings, similar to concepts and machines, get more advanced and complex as we become more compact and dense. We go to school to be more compact and denser, we go there to learn the “terminology” to terminate our notso-compact-lowlevel-elementaryschoolteacherimposed  thoughts.

Cumulant Generating Function applied on a Gamma Distribution

The Gamma distribution is given by;

$\displaystyle G(x|k,\theta)=\frac{x^{k-1}e^{-x/\theta}}{\theta^k \Gamma(k)} \ \ \ \ \ (1)$

We collect all the terms to the exponent, and have;

$\displaystyle G(x|k,\theta)=\frac{e^{(-x/\theta)+(k-1)\ln x}}{e^{k \ln \theta + \ln \Gamma(k)}} \ \ \ \ \ (2)$

Since we know this distribution to be a valid one, from the axioms of probability theory, we know that, if we integrate this expression with respect to x in ${(0, \infty)}$, we must have one. And we observe that the denominator is a constant, which can’t be of any use other than being a normalization constant. So we define a function,

$\displaystyle A(\alpha_1, \alpha_2)=\ln \int e^{(-x/\theta)+(k-1)\ln x}dx \ \ \ \ \ (3)$

where; ${\alpha_1=-1/\theta}$; ${\alpha_2 = k-1}$. Note that, the integral inside the logarithm gives the normalization constant. The latter can be seen from equation (2), since the integral of the numerator must be equal to the denominator, which is the normalization constant itself. So, for example if we consider, ${\partial A / \partial \alpha_1}$, we get;

$\displaystyle \frac{\partial A(\alpha_1, \alpha_2)}{\partial d\alpha_1}=\frac { \int xe^{(-x/\theta)+(k-1)\ln x}dx} { \int e^{(-x/\theta)+(k-1)\ln x}dx} = E[x] \ \ \ \ \ (4)$

We find the expected value of x.But the thing is, we know a super-easy to differentiate expression for ${A}$, since it is the natural logarithm of the normalization constant;

$\displaystyle A(\alpha_1, \alpha_2)=\ln \Big\{ {e^{k \ln \theta + \ln \Gamma(k)}}\Big\}=k \ln \theta + \ln \Gamma(k) \ \ \ \ \ (5)$

So;

$\displaystyle E[x] = \frac{\partial A(\alpha_1, \alpha_2)}{\partial \alpha_1}= \frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial (-1/\theta)}=k\theta \ \ \ \ \ (6)$

and;

$\displaystyle E[\ln x] = \frac{\partial A(\alpha_1, \alpha_2)}{\partial d\alpha_2}= \frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial (k-1)}=\frac{\partial (k \ln \theta + \ln \Gamma(k))}{\partial k}= \ln \theta + \frac{\Gamma'(k)}{\Gamma(k)} \ \ \ \ \ (7)$

There’s a special name given for ${\frac{\Gamma'(k)}{\Gamma(k)}}$, which is digamma function. We denote it by ${\psi(k)}$. So, according to the convention, ${E[\ln x]= \ln \theta + \psi ( k)}$.