모르는 게 없는 것 같은 착각을 하곤 하지만

책을 다시 펴보는 것은 물론이거니와

심지어 내가 일전에 이해해서 정리했던 글을 읽다보면

내가 단단히 착각하고 있다는 사실을 깨닫게 된다.

물론 모든 내용을 완벽히 이해한 적은 없기 때문에

완전히 생소한 것은 기억을 더듬어 자료를 찾아봐도 대부분 정리된 것이 애초에 없다.

여러번 읽기 전에는 처음에 놓친 내용이 무엇인지 조차 알지 못하니

이래서 책을 여러번 읽어야 하는 건가 싶다.

그래서 갑자기 떠오른 빅쇼트의 시작 문구

"It ain't what you don't know that gets you into trouble. It's what you know for sure that just ain't so.", a fake Mark Twain quote used in The Big Short

불현듯 어느 가사가 충렬왕으로 들렸다

그리고 하루종일 그 알 수없는 노래의 알수 없는 부분이 머리를 맴돌았다 뭐 정ㅎ학히는 충렬왕이라는 딘어가 말이다

무슨노래 였을까;

참고로 내머리에 남은 고려는 정도전 쌍화점 뿐이다;

국사 공부와 영어 공부가 동시에 필요한 현상이었다.

휴가철 이기도 하고 상황도 그렇고 해서 반나절을 작정하고 아무 것도 하지 않은채

대구로 출발했다

끝

-------------------------------------------------------------------

https://lingpipe-blog.com/2013/04/12/generative-vs-discriminative-bayesian-vs-frequentist/

-------------------------------------------------------------------

Generative vs. Discriminative; Bayesian vs. Frequentist

I was helping Boyi Xie get ready for his Ph.D. qualifying exams in computer science at Columbia and at one point I wrote the following diagram on the board to lay out the generative/discriminative and Bayesian/frequentist distinctions in what gets modeled.

To keep it in the NLP domain, let’s assume we have a simple categorization problem to predict a category z for an input consisting of a bag of words vector w and parameters β.

If you’re not familiar with frequentist notation, items to the right of the semicolon (;) are not modeled probabilistically.

Frequentists model the probability of observations given the parameters. This involves a likelihood function.

Bayesians model the joint probability of data and parameters, which, given the chain rule, amounts to a likelihood function times a prior.

Generative models provide a probabilistic model of the predictors, here the words w, and the categories z, whereas discriminative models only provide a probabilistic model of the categories z given the words w.

----------------------------------------------------------------------------

https://lingpipe-blog.com/2013/05/23/all-bayesian-models-are-generative-in-theory/

----------------------------------------------------------------------------

All Bayesian Models are Generative (in Theory)

I had a brief chat with Andrew Gelman about the topic of generative vs. discriminative models. It came up when I was asking him why he didn’t like the frequentist semicolon notation for variables that are not random. He said that in Bayesian analyses, all the variables are considered random. It just turns out we can sidestep having to model the predictors (i.e., covariates or features) in a regression. Andrew explains how this works in section 14.1, subsection “Formal Bayesian justification of conditional modeling” in Bayesian Data Analysis, 2nd Edition (the 3rd Edition is now complete and will be out in the foreseeable future). I’ll repeat the argument here.

Suppose we have predictor matrix X and outcome vector y. In regression modeling, we assume a coefficient vector \beta and explicitly model the outcome likelihood p(y|\beta,X) and coefficient prior p(\beta). But what about X? If we were modeling X, we’d have a full likelihood p(X,y|\beta,\psi), where \psi are the additional parameters involved in modeling X and the prior is now joint, p(\beta,\psi).

So how do we justify conditional modeling as Bayesians? We simply assume that \psi and \beta have independent priors, so that

p(\beta,\psi) = p(\beta) \times p(\psi).

The posterior then neatly factors as

p(\beta,\psi|y,X) = p(\psi|X) \times p(\beta|X,y).

Looking at just the inference for the regression coefficients \beta, we have the familiar expression

p(\beta|y,X) \propto p(\beta) \times p(y|\beta,X).

Therefore, we can think of everything as a joint model under the hood. Regression models involve an independence assumption so we can ignore the inference for \psi. To quote BDA,

The practical advantage of using such a regression model is that it is much easier to specify a realistic conditional distribution of one variable given k others than a joint distribution on all k+1 variables.

We knew that all along.

----------------------------

수식이 조금 깨지겠지만 우선..