stimmenfryslan/Readme.md

# Stimmen Fryslan

## Reproducibiliy results [paper xyz]

These notebooks allow for the reroducabiluty, they require access to the stimmen mysql database. One needs to request to this database.

### General statistics

[Statistics for Nanna's email of 2019-02-13](notebooks/Statistics%20for%20Nanna's%20email%20of%202019-02-13.ipynb)

Calculates statistics of the stimmen app usage.

### Regions

[Partition provinces in wijken and gemeentes](notebooks/Segment%20Provinces%20in%20Wijken%20and%20Gemeentes.ipynb)


Partitions Fryslan, the Dutch province, with repesct to two granularities, as defined by the CBS 'wijken' and 'gemeentes' of 2017. These partitionings are used in all maps created with the other notebooks.

### Heatmaps

[Frysian pronunciation occurrence](notebooks/Frysian pronunciation occurrence.ipynb)


Creates all heatmaps illustrating the distribution of one pronunciation relative to all other pronunciations of that word.

**Result example:**

Several pronunciation heatmaps maps for the word zaterdag, respectively across both rows for gemeentes and wijken granularities.

| snjoun | sa:tədex | sɑ:təɾjə | snɪwn |
|---|---|---|---|
| ![example pronunciation occurence map](images/heatmaps/gemeentes_zaterdag_snjoun.png) | ![example pronunciation occurence map](images/heatmaps/gemeentes_zaterdag_sa:tədex.png) | ![example pronunciation occurence map](images/heatmaps/gemeentes_zaterdag_sɑ:təɾjə.png) | ![example pronunciation occurence map](images/heatmaps/gemeentes_zaterdag_snɪwn.png) |
| ![example pronunciation occurence map](images/heatmaps/wijken_zaterdag_snjoun.png) | ![example pronunciation occurence map](images/heatmaps/wijken_zaterdag_sa:tədex.png) | ![example pronunciation occurence map](images/heatmaps/wijken_zaterdag_sɑ:təɾjə.png) | ![example pronunciation occurence map](images/heatmaps/wijken_zaterdag_snɪwn.png) |

### Distribution maps

Creates maps for both granularities, each illustrating the pronunciation distribution of one word.

[Frysian pronunciation distribution maps](notebooks/Frysian pronunciation distribution maps.ipynb)

**Result example:**

Several pronunciation distribution maps for different words, respectively across both rows for gemeentes and wijken granularities.

| zaterdag | vis | geel | oog |
|---|---|---|---|
| ![example pronunciation distribution map](images/bar-maps/gemeentes_zaterdag.png) | ![example pronunciation distribution map](images/bar-maps/gemeentes_vis.png) | ![example pronunciation distribution map](images/bar-maps/gemeentes_geel.png) | ![example pronunciation distribution map](images/bar-maps/gemeentes_oog.png) |
| ![example pronunciation distribution map](images/bar-maps/wijken_zaterdag.png) | ![example pronunciation distribution map](images/bar-maps/wijken_vis.png) | ![example pronunciation distribution map](images/bar-maps/wijken_geel.png) | ![example pronunciation distribution map](images/bar-maps/wijken_oog.png) |


## Notebooks

### Extract Frysian dialect regions

[notebook](notebooks/Dialect%20Regions%20from%20image.ipynb)

Get polygons of dialect regions as mapped in this image

![dialect regions](data/dialects.png)

using image processing.


Results

 * [data/frysian_dialect_regions.geojson](data/frysian_dialect_regions.geojson)

### Group recordings to Frysian dialect regions

[notebook](notebooks/Group%20recordings%20in%204%20Frysian%20dialect%20regions.ipynb)

Create spreadsheets with the recordings assigned to dialect regions.

Results

 * [data/picture_game_recordings_by_dialect.csv](data/picture_game_recordings_by_dialect.csv)
 * [data/picture_game_recordings_by_dialect.csv](data/picture_game_recordings_by_dialect.xlsx)
 * [data/free_speech_recordings_by_dialect.xlsx](data/free_speech_recordings_by_dialect.csv)
 * [data/free_speech_recordings_by_dialect.xlsx](data/free_speech_recordings_by_dialect.xlsx)

### Segment Friesland (and Groningen) in Gemeentes and Wijken

[notebook](notebooks/Segment%20Provinces%20in%20Wijken%20and%20Gemeentes.ipynb)

Some of the wijken are merges, for example part of Leeuwarden, to avoid that the segementation gets too fine
grained.

[notebook](notebooks/Show%20Province%20Segmentations.ipynb)

Visualized maps of the segmentations.

Results:

 * data/Friesland_gemeentes.geojson
 * data/Friesland_gemeentes.kml
 * data/Friesland_wijken.geojson
 * data/Friesland_wijken.kml
 * data/Groningen_gemeentes.geojson
 * data/Groningen_gemeentes.kml
 * data/Groningen_wijken.geojson
 * data/Groningen_wijken.kml

### Posterior probabilities and Likelyhoods for origin based on word pronunciation

[notebook](notebooks/Posterior%20probabilities%20and%20Likelyhoods%20for%20origin%20based%20on%20word%20pronunciation.ipynb)

Tables with the posterior probabilities and likelihoods of *being from a region* based on the *stated pronunciation of one specific word*.

### Gabmap tab seperated files

[notebook](notebooks/Gabmap%20Pronunciation%20Tables.ipynb)

Create tab separated files to be used by gapmap, based on the geojson regions as created by

`Segment Provinces in Wijken and Gemeentes.ipynb`

This is a simple example for the created gabmap files.

[notebook](notebooks/Gabmap%20Pronunciation%20Tables%2C%20Simple%20Example.ipynb)

 * [centroids](data/Centroids_example.gabmap.tsv)
 * [geojson](data/Gabmap_example.geojson)
 * [percentages](data/Pronunciation_percentages_example.gabmap.tsv)
 * [pronunciation](data/Pronunciations_example.gabmap.tsv)

### Bar Maps per word for Pronunciation Occurrence in Frysian Municipalities

For each word, a map illustrates the pronunciation occurrence as measured by the prediction quiz, per Frysian
municipality.

[notebook](notebooks/Bar%20Maps%20per%20word%20for%20Pronunciation%20Occurrence%20in%20Frysian%20Municipalities.ipynb)

### Heatmap per word for Pronunciation Occurrence in Frysian Municipalities

[notebook](notebooks/Heatmap%20per%20word%20for%20Pronunciation%20Occurrence%20in%20Frysian%20Municipalities.ipynb)

Each map displays the pronounciation occurence in Frysian municipalities for one word. Each pronunciation is represented by one map layer, and for one municipality layer the percentages for each pronunciation add up to 100% + rounding errors.

### Heatmap per word for Pronunciation Occurrence in Frysian Neighborhoods

Same as for Municipalities, but for Neighborhoods.

[notebook](notebooks/Heatmap%20per%20word%20for%20Pronunciation%20Occurrence%20in%20Frysian%20Neighborhoods.ipynb)