CANaerospace

CANaerospace is a higher layer protocol based on Controller Area Network (CAN) which has been developed by Stock Flight Systems in 1998 for aeronautical applications. == Background == CANaerospace supports airborne systems employing the Line-replaceable unit (LRU) concept to share data across CAN and ensures interoperability between CAN LRUs by defining CAN physical layer characteristics, network layers, communication mechanisms, data types and aeronautical axis systems. CANaerospace is an open source project, was initiated to standardize the interface between CAN LRUs on the system level. CANaerospace is continuously being developed further and has also been published by NASA as the Advanced General Aviation Transport Experiments Databus Standard in 2001. It found widespread use in aeronautical research worldwide. A major research aircraft that employs several CANaerospace networks for real-time computer interconnection is the Stratospheric Observatory for Infrared Astronomy (SOFIA), a Boeing 747SP with a 2.5m astronomic telescope. CANaerospace is also frequently used in flight simulation and connects entire aircraft cockpits (i.e. in Eurofighter Typhoon simulators) to the simulation host computers. In Italy CANaerospace is used as UAV data bus technology. Furthermore, CANaerospace serves as communication network in several general aviation avionics systems. The CANaerospace interface definition closes the gap between the ISO/OSI layer 1 and 2 CAN protocol (which is implemented in the CAN controller itself) and the specific requirements of distributed systems in aircraft. It may be used as a primary or ancillary avionics network and was designed to meet the following requirements: Democratic network: CANaerospace does not require any master/slave relationships between LRUs or a "bus controller", thereby avoiding a potential single source of failure. Every node in the network has the same rights for participation in the bus traffic. Self-identifying message format: Each CANaerospace message contains information about the type of the data and the transmitting node. This allows the data to be unambiguously recognized at each receiving node. Continuous Message Numbering: Each CANaerospace message contains a continuously incremented number which allows coherent processing of messages in the receiving stations. Message Status Code: Each CANaerospace message contains information about the integrity of the data is conveying. This allows receiving stations to evaluate the quality of the received data and to react accordingly. Emergency Event Signaling: CANaerospace defines a mechanism that allows each node to transmit information about exception or error situations. This information can be used by other stations to determine the network health. Node Service Interface: As an enhancement to CAN, CANaerospace provides a means for individual stations on the network to communicate with each other using connection-oriented and connectionless services. Predefined CAN Identifier Assignment: CANaerospace offers a predefined identifier assignment list for normal operation data. In addition to the predefined list, user-defined identifier assignment lists may be used. Ease of Implementation: The amount of code to implement CANaerospace is very little by design in order to minimize the effort for testing and certification of flight safety critical systems. Openness to Extensions: All CANaerospace definitions are extendable to provide flexibility for future enhancements and to allow adaptions to the requirements of specific applications. Free Availability: No cost whatsoever apply for the use of CANaerospace. The specification can be downloaded from the Internet == Physical interface == To ensure interoperability and reliable communication, CANaerospace specifies the electrical characteristics, bus transceiver requirements and data rates with the corresponding tolerances based on ISO 11898. The bit timing calculation (baud rate accuracy, sample point definition) and robustness to electromagnetic interference are given special emphasis. Also addressed are CAN connector, wiring considerations and design guidelines to maximize electromagnetic compatibility. == Communication layers == The Bosch CAN specification itself allows messages being transmitted both periodically and aperiodically but does not cover issues like data representation, node addressing or connection-oriented protocols. CAN is entirely based on Anyone-to-Many (ATM) communication which means that CAN messages are always received by all stations in the network. The advantage of the CAN concept is inherent data consistency between all stations, the drawback is that it does not allow node addressing which is the basis for Peer-to-Peer (PTP) communication. Using CAN networks in aeronautical applications, however, demands a standard targeted to the specific requirements of airborne systems which implies that communication between individual stations in the network must be possible to enable the required degree of system monitoring. Consequently, CANaerospace defines additional ISO/OSI layer 3, 4 and 6 functions to support node addressing and unified ATM/PTP communication mechanisms. PTP communication allows to set up client/server interactions between individual stations in the network either temporarily or permanently. More than one of these interactions may be in effect at any given time and each node may be client for one operation and server for another at the same time. This CANaerospace mechanism is called "Node Service Concept" and allows i.e. to distribute system functions over several stations in the network or to control dynamic system reconfiguration in case of failure. The Node Service concept supports both connection-oriented and connectionless interactions like with TCP/IP and UDP/IP for Ethernet. Enabling both ATM and PTP communication for CAN requires the introduction of independent network layers to isolate the different types of communication. This is realized for CANaerospace by forming CAN identifier groups as shown in Figure 1. The resulting structure creates Logical Communication Channels (LCCs) and assigns a specific communication type (ATM, PTP) to each of the LCCs. User-defined LCCs provide the necessary freedom for designers and allow the implementation of CANaerospace according to the needs of specific applications. Figure 1: Logical Communication Channels for CANaerospace As a side effect, the CAN identifier groups in Figure 1 affect the priority of the message transmission in case of bus arbitration. The communication channels are therefore arranged according to their relative importance: Emergency Event Data Channel (EED): This communication channel is used for messages which require immediate action (i.e. system degradation or reconfiguration) and have to be transmitted with very high priority. Emergency Event Data uses ATM communication exclusively. High/Low Priority Node Service Data Channel (NSH/NSL): These communication channels are used for client/server interactions using PTP communication. The corresponding services may be of the connection-oriented as well as the connectionless type. NSH/NSL may also be used to support test and maintenance functions. Normal Operation Data Channel (NOD): This communication channel is used for the transmission of the data which is generated during normal system operation and described in the CANaerospace identifier assignment list. These messages may be transmitted periodically or aperiodically as well as synchronously or asynchronously. All messages which cannot be assigned to other communication channels shall use this channel. High/Low Priority User-Defined Data Channel (UDH/UDL): This channel is dedicated to communication which cannot, due to their specific characteristics, be assigned other channels without violating the CANaerospace specification. As long as the defined identifier range is used, the message content and the communication type (ATM, PTP) for these channels may be specified by the system designer. To ensure interoperability it is highly recommended that the use of these channels is minimized. Debug Service Data Channel (DSD): This channel is dedicated to messages which are used temporarily for development and test purposes only and are not transmitted during normal operation. As long as the defined identifier range is used, the message content and the communication type (ATM, PTP) for these channels may be specified by the system designer. == Data representation == The majority of the real-time control systems used in aeronautics employ "big endian" processor architectures. This data representation was therefore specified for CANaerospace as well. With big endian data representation, the most significant bit of any datum is arranged leftmost and transmitted first on CANaerospace as shown in Figure 2. Figure 2: "Big Endian" Data Representation for CANaerospace CANaerospace uses a self-identifying message

BERT (language model)

Bidirectional encoder representations from transformers (BERT) is a language model introduced in October 2018 by researchers at Google. It learns to represent text as a sequence of vectors using self-supervised learning. It uses the encoder-only transformer architecture. BERT dramatically improved the state of the art for large language models. As of 2020, BERT is a ubiquitous baseline in natural language processing (NLP) experiments. BERT is trained by masked token prediction and next sentence prediction. With this training, BERT learns contextual, latent representations of tokens in their context, similar to ELMo and GPT-2. It found applications for many natural language processing tasks, such as coreference resolution and polysemy resolution. It improved on ELMo and spawned the study of "BERTology", which attempts to interpret what is learned by BERT. BERT was originally implemented in the English language at two model sizes, BERTBASE (110 million parameters) and BERTLARGE (340 million parameters). Both were trained on the Toronto BookCorpus (800M words) and English Wikipedia (2,500M words). The weights were released on GitHub. On March 11, 2020, 24 smaller models were released, the smallest being BERTTINY with just 4 million parameters. == Architecture == BERT is an "encoder-only" transformer architecture. At a high level, BERT consists of 4 modules: Tokenizer: This module converts a piece of English text into a sequence of integers ("tokens"). Embedding: This module converts the sequence of tokens into an array of real-valued vectors representing the tokens. It represents the conversion of discrete token types into a lower-dimensional Euclidean space. Encoder: a stack of Transformer blocks with self-attention, but without causal masking. Task head: This module converts the final representation vectors into one-shot encoded tokens again by producing a predicted probability distribution over the token types. It can be viewed as a simple decoder, decoding the latent representation into token types, or as an "un-embedding layer". The task head is necessary for pre-training, but it is often unnecessary for so-called "downstream tasks," such as question answering or sentiment classification. Instead, one removes the task head and replaces it with a newly initialized module suited for the task, and finetune the new module. The latent vector representation of the model is directly fed into this new module, allowing for sample-efficient transfer learning. === Embedding === This section describes the embedding used by BERTBASE. The other one, BERTLARGE, is similar, just larger. The tokenizer of BERT is WordPiece, which is a sub-word strategy like byte-pair encoding. Its vocabulary size is 30,000, and any token not appearing in its vocabulary is replaced by [UNK] ("unknown"). The first layer is the embedding layer, which contains three components: token type embeddings, position embeddings, and segment type embeddings. Token type: The token type is a standard embedding layer, translating a one-hot vector into a dense vector based on its token type. Position: The position embeddings are based on a token's position in the sequence. BERT uses absolute position embeddings, where each position in a sequence is mapped to a real-valued vector. Each dimension of the vector consists of a sinusoidal function that takes the position in the sequence as input. Segment type: Using a vocabulary of just 0 or 1, this embedding layer produces a dense vector based on whether the token belongs to the first or second text segment in that input. In other words, type-1 tokens are all tokens that appear after the [SEP] special token. All prior tokens are type-0. The three embedding vectors are added together representing the initial token representation as a function of these three pieces of information. After embedding, the vector representation is normalized using a LayerNorm operation, outputting a 768-dimensional vector for each input token. After this, the representation vectors are passed forward through 12 Transformer encoder blocks, and are decoded back to 30,000-dimensional vocabulary space using a basic affine transformation layer. === Architectural family === The encoder stack of BERT has 2 free parameters: L {\displaystyle L} , the number of layers, and H {\displaystyle H} , the hidden size. There are always H / 64 {\displaystyle H/64} self-attention heads, and the feed-forward/filter size is always 4 H {\displaystyle 4H} . By varying these two numbers, one obtains an entire family of BERT models. For BERT: the feed-forward size and filter size are synonymous. Both of them denote the number of dimensions in the middle layer of the feed-forward network. the hidden size and embedding size are synonymous. Both of them denote the number of real numbers used to represent a token. The notation for encoder stack is written as L/H. For example, BERTBASE is written as 12L/768H, BERTLARGE as 24L/1024H, and BERTTINY as 2L/128H. == Training == === Pre-training === BERT was pre-trained simultaneously on two tasks: Masked language modeling (MLM): In this task, BERT ingests a sequence of words, where one word may be randomly changed ("masked"), and BERT tries to predict the original words that had been changed. For example, in the sentence "The cat sat on the [MASK]," BERT would need to predict "mat." This helps BERT learn bidirectional context, meaning it understands the relationships between words not just from left to right or right to left but from both directions at the same time. Next sentence prediction (NSP): In this task, BERT is trained to predict whether one sentence logically follows another. For example, given two sentences, "The cat sat on the mat" and "It was a sunny day", BERT has to decide if the second sentence is a valid continuation of the first one. This helps BERT understand relationships between sentences, which is important for tasks like question answering or document classification. ==== Masked language modeling ==== In masked language modeling, 15% of tokens would be randomly selected for masked-prediction task, and the training objective was to predict the masked token given its context. In more detail, the selected token is: replaced with a [MASK] token with probability 80%, replaced with a random word token with probability 10%, not replaced with probability 10%. The reason not all selected tokens are masked is to avoid the dataset shift problem. The dataset shift problem arises when the distribution of inputs seen during training differs significantly from the distribution encountered during inference. A trained BERT model might be applied to word representation (like Word2Vec), where it would be run over sentences not containing any [MASK] tokens. It is later found that more diverse training objectives are generally better. As an illustrative example, consider the sentence "my dog is cute". It would first be divided into tokens like "my1 dog2 is3 cute4". Then a random token in the sentence would be picked. Let it be the 4th one "cute4". Next, there would be three possibilities: with probability 80%, the chosen token is masked, resulting in "my1 dog2 is3 [MASK]4"; with probability 10%, the chosen token is replaced by a uniformly sampled random token, such as "happy", resulting in "my1 dog2 is3 happy4"; with probability 10%, nothing is done, resulting in "my1 dog2 is3 cute4". After processing the input text, the model's 4th output vector is passed to its decoder layer, which outputs a probability distribution over its 30,000-dimensional vocabulary space. ==== Next sentence prediction ==== Given two sentences, the model predicts if they appear sequentially in the training corpus, outputting either [IsNext] or [NotNext]. During training, the algorithm sometimes samples two sentences from a single continuous span in the training corpus, while at other times, it samples two sentences from two discontinuous spans. The first sentence starts with a special token, [CLS] (for "classify"). The two sentences are separated by another special token, [SEP] (for "separate"). After processing the two sentences, the final vector for the [CLS] token is passed to a linear layer for binary classification into [IsNext] and [NotNext]. For example: Given "[CLS] my dog is cute [SEP] he likes playing [SEP]", the model should predict [IsNext]. Given "[CLS] my dog is cute [SEP] how do magnets work [SEP]", the model should predict [NotNext]. === Fine-tuning === BERT is meant as a general pretrained model for various applications in natural language processing. That is, after pre-training, BERT can be fine-tuned with fewer resources on smaller datasets to optimize its performance on specific tasks such as natural language inference and text classification, and sequence-to-sequence-based language generation tasks such as question answering and conversational response generation. The original BERT paper published results demonstrating that a small amount of fine

G.9970

G.9970 (also known as G.hnta) is a Recommendation developed by ITU-T that describes the generic transport architecture for home networks and their interfaces to a provider's access network. G.9970 was developed by Study Group 15, Question 1. G.9970 received Consent on December 12, 2008 and was Approved on January 13, 2009. == Relationship with G.hn == G.9970 (G.hnta) and G.9960 (G.hn) are two ITU-T Recommendations that address home networking in a complementary manner. While G.9970 addresses layer 3 (network layer) of the home network architecture, G.9960 addresses layers 1 (physical layer) and 2 (data link layer).

Public Services Network

The Public Services Network (PSN) is a UK government's high-performance network, which helps public sector organisations work together, reduce duplication and share resources. It unified the provision of network infrastructure across the United Kingdom public sector into an interconnected "network of networks" to increase efficiency and reduce overall public expenditure. It is now a legacy network and public sector organisations are being migrated to using services on the public internet. == Origins == The Public Services Network (PSN) was launched officially as part of the Transformational Government Strategy commencing in 2005, under the original name of the Public Sector Network. Prior to this, some parts of local government had already successfully implemented the concept. The Hampshire Public Services Network (HPSN) was the first PSN, launched in 1999, followed closely by Kent County Councils partnerships with the KPSN. The HPSN, encompassing all of the borough, district and unitary councils, with the County Council, as well as the Fire Services, the Isle of Wight Council and 540 schools. National PSN technical and architecture compliance criteria were established from 2007, by GDS working with local government leaders from Socitm (the Society of Information Technology Management) on the National CIO Council and the Local CIO Council. The PSN's aim was to bring public services organisations with a common interest onto a single, coherent and standards-based ‘network of networks’. This would create influence, economies of scale and a commonality of standards for secure and easy inter-connection between public service organisations. The original concept of a network of networks strategy was based upon the work already undertaken in local government and recognition of Communities of Interest (COI) within the Criminal Justice Sector during work by the Office for Criminal Justice Reform (OCJR) between 2005 and 2007 to enable data sharing across business units. In this context a COI was defined as groups of Government departments and external partners who in combination provided services within a specific area of operation and used the same data, with a similar risk profile, shared risk appetite and common governance framework. Historically each group member had implemented their own networks and standards of operation in isolation with little or no consideration as to how services and data may be shared and resulting in increased costs of operation. The Network of Networks strategy proposed within OCJR recommended the creation of specific networks based upon these Communities of Interest which were joined together through data interchange gateways supporting common standards. Under this approach networks would be arranged by data type and business functions such as Criminal Justice, Health and Social Care, Defence and Intelligence or Public Finance rather than solely on established departmental boundaries. Within a COI, trust relationships and data interchange are readily supported, enabling data sharing without a need to cross network boundaries and providing benefits of scale without the challenges and compromises intrinsic to homogeneous cross sector networks. Data is made available without a need to transport it between organisations and control is retained by the data originator. In early 2007 a group of UK Government department CTOs in conjunction with the Office for Government Commerce Buying Solutions (OGC BS) established the vision for a single commonly provided, procured and managed public sector voice and data network infrastructure to replace the multitude of separately procured and managed networks serving various segments of the UK public sector; Education, Health, Central Government, Local Government etc. In 2008 an Industry Working Group was established to document the objectives and requirements more clearly. Their report set out the architectural and commercial principles as well as anticipated security, service management, governance and transition arrangements. == Architecture == The PSN comprises a core network, the Government Conveyancing Network or GCN provided by GCN Service Providers or GCNSPs. The GCN interconnects multiple operator networks, termed Direct Network Service Providers or DNSPs. Subscriber organisations contract to a connection from a local participating DNSP, connect via that to GCN and hence onwards to other interconnected networks and services. The GCN network is entirely based on IPv4 and MPLS and the GCNSPs are not currently mandated to provide IPv6, though they should have a roadmap to implementing it if and when required. == Commercial framework == In 2010 Virgin Media Business, BT, Cable & Wireless and Global Crossing signed Deeds of Undertaking (DoU) and subsequently achieved accreditation for providing GCN and IP VPN services. In March 2012, BT, Cable & Wireless, Capita Business Services, Eircom, Fujitsu, Kcom, Level 3, Logicalis, MDNX, Thales, Updata and Virgin Media Business were successful bidders for the initial two-year PSN Connectivity framework. In June 2012, 29 companies were confirmed as suppliers of ICT services to the UK public sector under the Government's PSN Services framework contract. Apart from most of the previous suppliers, additional companies also included 2e2, Airwave Solutions, Azzurri Communications, Cassidian, CSC Computer Sciences, Computacenter, Daisy Communications, Easynet Global Services, EE, Freedom Communications, Icom Holdings, NextiraOne, PageOne Communications, Phoenix IT Group, Siemens Communications, Specialist Computer Centres, Telefónica, telent Technology Services, Uniworld Communications and Vodafone. == Governance == The PSN is managed within the Cabinet Office where it is part of the Government Digital Service. == Early implementations == There were already notable initiatives in progress in county council areas, demonstrating public sector network integration in both the Hampshire HPSN2 network and in Kent's community network. Project Pathway was established as a pilot linking these two county-wide networks, with Virgin Media Business and Global Crossing the subscriber and GCN network elements. Staffordshire County Council was the first council in England to establish a PSN that included the county's NHS Health partners. Other county councils have since followed the leads of these councils. == Transition == Centrally procured public sector networks are expected to migrate across to the PSN framework as they reach the end of their contract terms, either through an interim framework or directly. The Government Secure Intranet (GSi) contracts expired in September 2011, running on to 12 February 2012 and were replaced by the transitional Government Secure Intranet Convergence Framework (GCF). The Managed Telephony Service (MTS) contract expired on 31 December 2011 and was replaced by the Managed Telephony Convergence Framework (MTCF). == Future plan == In a blog post published on 20 January 2017, Government Digital Service announced that the Technology Leaders Network (TLN) had agreed that government was starting a journey away from the PSN. This was because using the Internet was considered suitable for the vast majority of the work that the public sector does. The blog post confirmed that the 'move was not going to happen immediately' and stated that 'there's quite a bit of work to do across the public sector to prepare for the changes'. It also stated that it was too early for a full timeline to be provided, although all PSN-connected organisations would be updated as the process evolved. The blog post confirmed that organisations that need to access services that are only available on the PSN would still need to connect to it for the time being and continue to meet its assurance requirements. In a blog post published on 16 March 2017, Government Digital Service (GDS) set out its plans for PSN assurance. The blog post confirmed that the PSN compliance process wasn't 'going anywhere, certainly for a while yet'. It explained that the TLN agreed that – as one of the only recognised, externally accredited, cross-government common assurance standards – it 'needs to live on far beyond the end of the physical PSN network'. Government Digital Service, along with the National Cyber Security Centre (NCSC) and the Cyber and Government Security Directorate, are now looking at ways to expand and reframe PSN compliance in a new context that, while retaining the assurance principles that are the basis of the existing process, will aim to improve the process. A GDS blog post titled 'The road to closing down the PSN' published on 8 September 2020 describes how the public sector will migrate away from the PSN. The Cabinet Office has set up a programme called Future Networks for Government (FN4G) to help organisations move away from the PSN.

Torus interconnect

A torus interconnect is a switch-less network topology for connecting processing nodes in a parallel computer system. == Introduction == In geometry, a torus is created by revolving a circle about an axis coplanar to the circle. While this is a general definition in geometry, the topological properties of this type of shape describes the network topology in its essence. === Geometry illustration === In the representations below, the first is a one dimension torus, a simple circle. The second is a two dimension torus, in the shape of a 'doughnut'. The animation illustrates how a two dimension torus is generated from a rectangle by connecting its two pairs of opposite edges. At one dimension, a torus topology is equivalent to a ring interconnect network, in the shape of a circle. At two dimensions, it becomes equivalent to a two dimension mesh, but with extra connection at the edge nodes. === Torus network topology === A torus interconnect is a switch-less topology that can be seen as a mesh interconnect with nodes arranged in a rectilinear array of N = 2, 3, or more dimensions, with processors connected to their nearest neighbors, and corresponding processors on opposite edges of the array connected.[1] In this lattice, each node has 2N connections. This topology is named for the lattice formed in this way, which is topologically homogeneous to an N-dimensional torus. == Visualization == The first 3 dimensions of torus network topology are easier to visualize and are described below: 1D Torus: one dimension, n nodes are connected in closed loop with each node connected to its two nearest neighbors. Communication can take place in two directions, +x and −x. A 1D Torus is the same as ring interconnection. 2D Torus: two dimensions with degree of four, the nodes are imagined laid out in a two-dimensional rectangular lattice of n rows and n columns, with each node connected to its four nearest neighbors, and corresponding nodes on opposite edges connected. Communication can take place in four directions, +x, −x, +y, and −y. The total nodes of a 2D Torus is n2. 3D Torus: three dimensions, the nodes are imagined in a three-dimensional lattice in the shape of a rectangular prism, with each node connected with its six neighbors, with corresponding nodes on opposing faces of the array connected. Each edge consists of n nodes. communication can take place in six directions, +x, −x, +y, −y, +z, −z. Each edge of a 3D Torus consist of n nodes. The total nodes of 3D Torus is n3. ND Torus: N dimensions, each node of an N dimension torus has 2N neighbors, Communication can take place in 2N directions. Each edge consists of n nodes. Total nodes of this torus is nN. The main motivation of having higher dimension of torus is to achieve higher bandwidth, lower latency, and higher scalability. Higher-dimensional arrays are difficult to visualize. The above ruleset shows that each higher dimension adds another pair of nearest neighbor connections to each node. == Performance == A number of supercomputers on the TOP500 list use three-dimensional torus networks, e.g. IBM's Blue Gene/L and Blue Gene/P, and the Cray XT3. IBM's Blue Gene/Q uses a five-dimensional torus network. Fujitsu's K computer and the PRIMEHPC FX10 use a proprietary three-dimensional torus 3D mesh interconnect called Tofu. === 3D Torus performance simulation === Sandeep Palur and Dr. Ioan Raicu from Illinois Institute of Technology conducted experiments to simulate 3D torus performance. Their experiments ran on a computer with 250GB RAM, 48 cores and x86_64 architecture. The simulator they used was ROSS (Rensselaer’s Optimistic Simulation System). They mainly focused on three aspects: Varying network size Varying number of servers Varying message size They concluded that throughput decreases with the increase of servers and network size. Otherwise, throughput increases with the increase of message size. === 6D Torus product performance === Fujitsu Limited developed a 6D torus computer model called "Tofu". In their model, a 6D torus can achieve 100 GB/s off-chip bandwidth, 12 times higher scalability than a 3D torus, and high fault tolerance. The model is used in the K computer and Fugaku. === Cost === While long wrap-around links may be the easiest way to visualize the connection topology, in practice, restrictions on cable lengths often make long wrap-around links impractical. Instead, directly connected nodes—including nodes that the above visualization places on opposite edges of a grid, connected by a long wrap-around link—are physically placed nearly adjacent to each other in a folded torus network. Every link in the folded torus network is very short—almost as short as the nearest-neighbor links in a simple grid interconnect—and therefore low-latency.

Second-order co-occurrence pointwise mutual information

In computational linguistics, second-order co-occurrence pointwise mutual information (SOC-PMI) is a method used to measure semantic similarity, or how close in meaning two words are. The method does not require the two words to appear together in a text. Instead, it works by analyzing the "neighbor" words that typically appear alongside each of the two target words in a large body of text (corpus). If the two target words frequently share the same neighbors, they are considered semantically similar. For example, the words "cemetery" and "graveyard" may not appear in the same sentence often, but they both frequently appear near words like "buried," "dead," and "funeral." SOC-PMI uses this shared context to determine that they have a similar meaning. The method is called "second-order" because it doesn't look at the direct co-occurrence of the target words (which would be first-order), but at the co-occurrence of their neighbors (a second level of association). The strength of these associations is quantified using pointwise mutual information (PMI). == History == The method builds on earlier work like the PMI-IR algorithm, which used the AltaVista search engine to calculate word association probabilities. The key advantage of a second-order approach like SOC-PMI is its ability to measure similarity between words that do not co-occur often, or at all. The British National Corpus (BNC) has been used as a source for word frequencies and contexts for this method. == Methodology == The SOC-PMI algorithm measures the similarity between two words, w 1 {\displaystyle w_{1}} and w 2 {\displaystyle w_{2}} , in several steps. === Step 1: Score neighboring words with PMI === First, for each target word ( w 1 {\displaystyle w_{1}} and w 2 {\displaystyle w_{2}} ), the algorithm identifies its "neighbor" words within a certain text window (e.g., within 5 words to the left or right) across a large corpus. The strength of the association between a target word t i {\displaystyle t_{i}} and its neighbor w {\displaystyle w} is calculated using pointwise mutual information (PMI). A higher PMI value means the two words appear together more often than would be expected by chance. The PMI between a target word t i {\displaystyle t_{i}} and a neighbor word w {\displaystyle w} is calculated as: f pmi ( t i , w ) = log 2 ⁡ f b ( t i , w ) × m f t ( t i ) f t ( w ) {\displaystyle f^{\text{pmi}}(t_{i},w)=\log _{2}{\frac {f^{b}(t_{i},w)\times m}{f^{t}(t_{i})f^{t}(w)}}} where: f b ( t i , w ) {\displaystyle f^{b}(t_{i},w)} is the number of times t i {\displaystyle t_{i}} and w {\displaystyle w} appear together in the context window. f t ( t i ) {\displaystyle f^{t}(t_{i})} is the total number of times t i {\displaystyle t_{i}} appears in the corpus. f t ( w ) {\displaystyle f^{t}(w)} is the total number of times w {\displaystyle w} appears in the corpus. m {\displaystyle m} is the total number of tokens (words) in the corpus. === Step 2: Create a semantic 'signature' for each word === For each target word ( w 1 {\displaystyle w_{1}} and w 2 {\displaystyle w_{2}} ), the algorithm creates a list of its most significant neighbors. This is done by taking the top β {\displaystyle \beta } neighbor words, sorted in descending order by their PMI score with the target word. This list of top neighbors, X w {\displaystyle X^{w}} , acts as a semantic "signature" for the word w {\displaystyle w} . X w = { X i w } {\displaystyle X^{w}=\{X_{i}^{w}\}} , for i = 1 , 2 , … , β {\displaystyle i=1,2,\ldots ,\beta } The size of this list, β {\displaystyle \beta } , is a parameter of the method. === Step 3: Compare the signatures === The algorithm then compares the signatures of w 1 {\displaystyle w_{1}} and w 2 {\displaystyle w_{2}} . It looks for words that are present in both signatures. The similarity of w 1 {\displaystyle w_{1}} to w 2 {\displaystyle w_{2}} is calculated by summing the PMI scores of w 2 {\displaystyle w_{2}} with every word in w 1 {\displaystyle w_{1}} 's signature list. The β {\displaystyle \beta } -PMI summation function defines this score. The score for w 1 {\displaystyle w_{1}} with respect to w 2 {\displaystyle w_{2}} is: f ( w 1 , w 2 , β ) = ∑ i = 1 β ( f pmi ( X i w 1 , w 2 ) ) γ {\displaystyle f(w_{1},w_{2},\beta )=\sum _{i=1}^{\beta }(f^{\text{pmi}}(X_{i}^{w_{1}},w_{2}))^{\gamma }} This sum only includes terms where the PMI value is positive. The exponent γ {\displaystyle \gamma } (with a value > 1) is used to give more weight to neighbors that are more strongly associated with w 2 {\displaystyle w_{2}} . This calculation is done in both directions: The similarity of w 1 {\displaystyle w_{1}} with respect to w 2 {\displaystyle w_{2}} : f ( w 1 , w 2 , β 1 ) = ∑ i = 1 β 1 ( f pmi ( X i w 1 , w 2 ) ) γ {\displaystyle f(w_{1},w_{2},\beta _{1})=\sum _{i=1}^{\beta _{1}}(f^{\text{pmi}}(X_{i}^{w_{1}},w_{2}))^{\gamma }} The similarity of w 2 {\displaystyle w_{2}} with respect to w 1 {\displaystyle w_{1}} : f ( w 2 , w 1 , β 2 ) = ∑ i = 1 β 2 ( f pmi ( X i w 2 , w 1 ) ) γ {\displaystyle f(w_{2},w_{1},\beta _{2})=\sum _{i=1}^{\beta _{2}}(f^{\text{pmi}}(X_{i}^{w_{2}},w_{1}))^{\gamma }} === Step 4: Calculate final similarity score === Finally, the total semantic similarity is the average of the two scores from the previous step. S i m ( w 1 , w 2 ) = f ( w 1 , w 2 , β 1 ) β 1 + f ( w 2 , w 1 , β 2 ) β 2 {\displaystyle \mathrm {Sim} (w_{1},w_{2})={\frac {f(w_{1},w_{2},\beta _{1})}{\beta _{1}}}+{\frac {f(w_{2},w_{1},\beta _{2})}{\beta _{2}}}} This score can be normalized to fall between 0 and 1. For example, using this method, the words cemetery and graveyard achieve a high similarity score of 0.986 (with specific parameter settings).

Social Media (Age-Restricted Users) Bill

The Social Media (Age-Restricted Users) Bill is a member's bill by National Party Member of Parliament Catherine Wedd that seeks to ban children under the age of 16 years from accessing social media by forcing social media companies to implement age verification measures. It is modelled after the Australian government's Online Safety Amendment. In mid October 2025, the New Zealand Parliament confirmed plans to introduce the social media age restriction bill. == Background == In late November 2024, the Albanese government of Australia, with support from the opposition Coalition parties, passed the Online Safety Amendment creating a world-first age verification regime targeting social media platforms operating in the country. The ban targets several social media platforms including Facebook, Instagram, Kick, Reddit, Snapchat, Threads, TikTok, Twitch, X (formerly Twitter) and YouTube. These platforms were required to implement age verification systems and to remove under-age users by 10 December 2025, when the law change came into effect. == Draft provisions == The draft Social Media (Age-Restricted Users) Bill defines social media platforms as electronic platforms that enable social media interactions between two or more end-users, facilitates communication between multiple end-users and allows users to post content on the platform. The proposed bill requires social media companies to take action to prevent users under the age of 16 from creating accounts on their platforms. It also creates a framework for courts to impose fines on platforms that fail to take reasonable steps to prevent underaged users from accessing the platform. == Legislative history == === Draft legislation === On 6 May 2025, Wedd announced a private member's bill called the "Social Media (Age-Restricted Users) Bill" that would bar access to social media platforms for people under the age of 16 years. She said that she was motivated as the mother of four children to support families, parents and teachers' efforts to manage their children's online exposure and the passage of the Australian Online Safety Amendment legislation in December 2024. Since National's coalition partner ACT New Zealand had refused to support the bill, the Sixth National Government announce it as a member's bill rather than a government bill. Prime Minister Christopher Luxon has confirmed that National would seek cross-party support for the legislation. ACT MP and the Minister of Internal Affairs Brooke van Velden said that the Government would watch the implementation of the Australian social media age restriction policy. In October 2025, Wedd's bill was drawn from the parliamentary ballot. In addition, Labour Reuben Davidson drafted a similar member's bill that would hold social media providers responsible for restricting "harmful content" and imposed NZ$50,000 fines for non-compliance. In November 2025, Luxon reiterated his support for social media age restriction legislation and said the New Zealand government would introduce a bill in 2026 before the 2026 New Zealand general election. He also confirmed that Education Minister Erica Stanford was leading an investigation into what lessons could be learnt from the Australian legislation. At the request of ACT MP Parmjeet Parmar, Parliament's Education and Workforce Committee held an inquiry into a proposed social media ban in early October 2025. The committee was led by National MP Carl Bates and received 430 submissions from 400 groups and individuals. The committee also heard from 87 in-person submissions. On 10 December 2025, the committee made 12 recommendations including restricting social media access to persons under the age of 16, re-evaluating existing legislation such as the Films, Videos, and Publications Classification Act and the Harmful Digital Communications Act 2015, and regulating online platforms and Internet service providers. The ACT party released a dissenting view disagreeing with the need for a law restricting social media access to under-16 year olds. In mid-May 2026, the Government confirmed that work on the proposed bill to ban under-16 year olds from social media had been paused. The New Zealand Parliament held a debate on the proposed bill on 13 May following a select committee inquiry into the harms caused by social media platforms. While the opposition Labour Party has agreed to support the member's bill, the ACT and Green parties opposed the proposed bill on the grounds that the rules were easy to circumvent, that at-risk groups could become more isolated, and that social media also harmed other age groups. == Responses == === Academia and civil society === In late July 2025, the New Zealand Council for Civil Liberties (NZCCL) expressed concern that the proposed social media age restriction could infringe upon the New Zealand Bill of Rights Act 1990, the Privacy Act 2020 and the United Nations' Convention on the Rights of the Child. The NZCCL also questioned the practicality of age verification software, a social media age limit and whether it would fulfil its stated goal of combating online harm. In August 2025, University of Auckland criminologist and senior lecturer Claire Meehan expressed concern that the social media age restriction legislation would cut children from their friendship and support networks. She also said that children and young people were digital natives who could use VPNs to circumvent the ban. Similar sentiments were echoed by Victoria University of Wellington media and communications lecturer Alex Beattie and "Ocean Today" Instagram social media influencer "Charlie." In October 2025, New Zealand Initiative representative Dr Eric Crampton expressed concern that a social media age restriction would involve the introduction of digital IDs. He argued that a new law was unnecessary and said that parents could limit their children's exposure to social media via Google's Family Link and Apple's equivalent. Similarly, Institute of Economic Affairs public policy fellow Matthew Lesh and the British Free Speech Union expressed concerns that young people could use VPNs to circumvent a social media ban, citing the spike in VPN usage in the United Kingdom following the passage of the Online Safety Act 2023. The advocacy group B416's co-chair Anna Curzon advocated for a social media ban on underage users, stating that social media apps "are made to be addictive" and made it difficult for parents to relate with their children. In late November 2025, B416's co-founder Anna Mowbray expressed support for the Government's social media age restriction bill but expressed disappointment that Luxon had not timed his announcement with the launch of the group's campaign. Generation-Z Aotearoa co-founder Lola Fisher has called on the New Zealand Government to consult with young people on the development of the legislation. === Government agencies and departments === In early October 2025, Privacy Commissioner Michael Webster expressed concern that social media platforms requiring users to prove their age via digital IDs could raise privacy concerns. Webster suggested that age verification systems could relay on various documents including passports. He said that age estimation technologies had high error rates and that age inference technologies relied on data mining. === Political parties === In early May 2025, the National Party government expressed support for a social media age restriction legislation. By contrast, its coalition partner ACT has opposed such legislation. ACT leader David Seymour described the ban as hasty and unworkable since it did not involve parents. Meanwhile, New Zealand First leader Winston Peters expressed support for a social media age restriction but said the bill should be subject to a select committee inquiry. The opposition Labour Party leader Chris Hipkins has expressed interest in a social media age restriction legislation but emphasised the need for consensus. Meanwhile, Green Party co-leader Chlöe Swarbrick said she wanted to learn more about the bill but described it as simplistic. Fellow Greens co-leader Marama Davidson said that the proposed bill would punish children and young people for the harm caused by big tech platforms. === Tech companies === In early October 2025, representatives of TikTok and Meta Platforms cautioned against proposed social media ban on under-16 years olds. During a one-day parliamentary inquiry, Ella Woods-Joyce, TikTok's public policy lead for Australia and New Zealand, and Mia Garlick, Meta's regional director of policy, expressed concern that the social media age restriction could send children and young people to less regulated online spaces. Woods-Joyce highlighted TikTok's policy of closing down accounts belonging to users under the age of 13 years while Garlick highlighted Meta's policy of placing users under the age of 16 in private accounts by default. In early February 2026 Meta's vice president and global head of safety, Antigone Da