Using hierarchical Bayesian for subscript range prescriber

[hyunjimoon]

Nunzio and Zeynep decided to investigate the following question using tools taught in 15.879

• Could we use hierarchical Bayesian estimation to prescribe the optimal number of subscript range?
• Could this be viewed from Data-Analytical Lookup and allocation function and ? (data and parameter are symmetric)

The above may also be relevant to allocation function as it builds on subscript.

Regarding actual code development with @tomfid, #17 is the best place to keep track of.
Our initial discussion was Nunzio can provide agent and network based knowledge and Zeynep can contribute with real life example (perhaps it might be helpful if you briefly introduce your interest and major)

[hyunjimoon]

This is the visualization in Bayesian workflow paper and Tom and I had the following discussion:

Tom says:
a. one global region
b. individual countries, with hierarchical priors limiting variety among them
c. pooling into WHO superregions
d. pooling into PM2.5 clusters
e. individual countries, with a two-tiered hierarchy with low variety within superregions or clusters, and higher variety across those units
Option c or d is probably going to be the most practical, but I would expect e to outperform them, if it could be implemented.

and I answered:
In Bayes-term, a. is no-pooling, b. is partial-pooling.

Now, apologies for self-citation but with managerial usage in mind, I judged the most explainable form of time-series forecasting is cluster-based (mixture) and hierarchical Baye-based so I have been digging..

c, d is cluster-then complete-pooling; c. is knowledge-based and d. is data-driven. For d: Time Unit Clustering Model for Pallet Movement Amount (Korean, English abstract)

e. Hierarchical spline for time series prediction: An application to naval ship engine failure rate

Going one step further, combine c, d, e. As you imagined two cluster methods (WHO superregions in c. and PM2.5 in e.) for complete pooling, different subclusters can exist and we can aggregate the two cluster-based models for the final forecast. Weights of each model are estimated simultaneously with each model's parameter. This is f, but if the two clusters are seasonality (weekly and monthly) it can be better modeled which is g. All of them have replicable Stan code.

f. mixture of two, two-tier hierarchical models: Failure Function of Logistics Industrial Equipment Using Bayesian Aggregation: an Application to Electric Forklift Failure Rate

g. apply f. for multi-seasonal time-series: Mixed pooling of seasonality for time series forecasting: An application to pallet transport data

Summarized as diagram:

[zhasgul]

So I'll briefly introduce my background/ interests as well as current research. I am an Industrial engineer and Master's student. I mainly focus on system dynamics modeling for health care problems, and I am working as a graduate researcher at MGH/HMS. In the context of this discussion, I was interested as I am currently building a model of quality of life dynamics for patients. This is focused on new therapeutic and observed clinical outcomes for patients/patient groups in the past, changing doctors' willingness to recommend this therapy in the future. However, the quality of life of patients is not independent of their current conditions. And the outcome of the treatment is not the sole determiner of the quality of life. I thought it could be interesting to add individual patients to the model with different variables, such as comorbidities and social environment. I was thinking of doing this with 1 subscript and ie. 100 elements by varying the parameters for the elements. However, most likely there are patient clusters that present similar quality of life outcomes. And it would be interesting to capture the features and heterogeneities similar to the paper in visualization in the Bayesian workflow paper Angie has shared.

[tomfid]

I think a key question here is, what is the objective function? Purely predictive power? Or is there some benefit to simplicity & transparency, or some cost to complexity that needs to be traded off? Also, seems like there's a potential forking paths problem.

[hyunjimoon]

Remedy for forking path problem from Andrew' textbook ROS: endogenize variation and report
ResearcherDOF_ForkingPath.pdf

[hyunjimoon]

In the reliable EVC context, #142, objective can be minimizing the time to make 50% of new vehicle sales to EV or phase out ICE, based on Biden (50% of all new vehicle sales by 2030 to be EVs or plug-in hybrids) and California (phase out ICE by 2035) policy.

From an integer programming perspective, subscript whose integrality cannot be avoided even in continuous SD context is interesting.

For structured + data-driven decision making, IP modelers would feel excited if analogue of cutting plane an decomosition (two stage decision making) can be found in MIP problem where decision variable is the number of subscript. To be more concrete an algorithm that adaptively adjusts time T (year, month, day) and space S (different level of aggregation within California / or if U.S. is chosen, different group of states (~5 - based on EV policy), state (51), county (>100)) is the feasible set of integer. For brevity, let's fix T and start from S branching. Node selection, branching variable selection, finding cutting plane, mixing strategy for cutting and branching are problems to be addressed. Specific implementation will be developed in #142.

[hyunjimoon]

Trajectory clustering in travel behavior

takeaway on spatio-temporal analysis

[hyunjimoon]

future of work

[hyunjimoon]

[hyunjimoon]

to bump up dynamic aggregation topic + start from big to small number of sub-organization rather the other direction to minimize bias. which shares spirit with structural learning then parameter tuning

Based on our discussion on #34 ,

During the talk we prepared together (#156 ), @tomfid introduced and it may have some relation with #161 as m3s team were looking for privacy algorithm that keeps track of quantity of interest (economy, health) without disaggregating them down to individual level.

[hyunjimoon]

I asked @jeff-dotson on discrete choice modeling.

are you aware of PyBLP package which estimates the demand for differentiated products with BLP-type random coefficients logit models? Related to our idea of applying rational meaning construction to equity valuation, my friend from Harvard econ put PyBLP package on my radar. I verified her the relevance of my goal with discrete choice model. I don't yet have intuitive feelings about the relation between our framework and PyBLP. Nor can I judge whether looking into this package might help or harm our execution.

Jeff answered he's very familiar with BLP (including Bayesian variants) which was designed to allow for estimation of a distribution of individual heterogeneity from aggregate level data (i.e., market share data for auto sales). This is a top down approach (aggregate to individual) where the approach we are pursuing is bottom up (individuals to aggregate). I would describe the latter as micro-founded and, hence, more informative of the process we care about.

Jeff's comment on top down vs bottom up, made me think of this thread on dynamic aggregation i.e. using the inferred outcome of hierarchical bayesian to set the level of aggregation. It also reminded me, solving the level of aggregation (optimization problem) for time series is called middle out, as detailed here. Interesting top down is alway biased from the screenshot below.