— Ch. 1 · The First Neural Network Paper —
Transfer learning.
~3 min read · Ch. 1 of 5
In 1976, Stevo Bozinovski and Ante Fulgosi published a paper addressing transfer learning in neural network training. Their work appeared in the Proceedings of Symposium Informatica 3-121-5 held in Bled. The original text was written in Croatian before translation efforts began. This document provided both a mathematical model and a geometrical model for the topic. It marked the earliest known attempt to describe how knowledge from one task could help another. A decade later, Bozinovski released a reminder about this first paper in 2020 within the journal Informica. That retrospective piece spanned pages 291 through 302 of volume 44. By 1981, researchers tested these ideas on actual image datasets containing letters from computer terminals. These experiments showed that transfer learning could produce positive results or negative ones depending on the setup.
Domains And Predictive Functions
A domain consists of a feature space and a marginal probability distribution where the two elements interact. Given a specific domain labeled D_s, a task requires two components: a label space Y and an objective predictive function f. The function f predicts the corresponding label y of a new instance x. This task T_s is learned from training data consisting of pairs (x_i, y_i). When comparing a source domain D_s with a target domain D_t, transfer learning aims to improve the learning of the target predictive function f_t. The improvement happens using knowledge found in D_s and T_s. If D_s equals D_t but T_s differs from T_t, the method still applies. Alternatively, if T_s equals T_t but D_s differs from D_t, the approach remains valid. The goal always centers on boosting performance for the target task.Algorithms And Network Types
Algorithms for transfer learning exist within Markov logic networks and Bayesian networks. These structures allow systems to move information between different problem spaces. Researchers applied these methods to cancer subtype discovery during the 32nd Conference on Neural Information Processing Systems in Montréal. That event took place in 2018 and featured work by Hajiramezanali, Dadaneh, Karbalayghareh, Zhou, and Qian. Other applications included building utilization and general game playing as described in a 2007 IJCAI paper by Bikramjit Banerjee and Peter Stone. Text classification tasks also benefited from these techniques alongside digit recognition efforts. Medical imaging projects used similar strategies while spam filtering operations adopted them too. Convolutional neural networks showed improved accuracy when exposed to another domain before any formal learning began. Results reached higher asymptotes at the end of the process compared to standard random weight distribution.