### Install R Package Manager (pacman)
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2015/05/20/data-visualisation-with-r
This R code snippet demonstrates how to install the 'pacman' package from CRAN. Pacman is a package manager that simplifies the installation and loading of other R packages, streamlining the setup process for R projects.
```R
install.packages("pacman")
```
--------------------------------
### Example Spark Core API Temperature GET Request
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2015/01/11/soda-sensor-push
A concrete example of a GET request URL to fetch the 'temperature' variable from a specific Spark Core device. An access token is required for authentication.
```APIDOC
https://api.spark.io/v1/devices/48ff6b065067555044472287/temperature?access_token={myaccesstoken}
```
--------------------------------
### Install Python Data Analysis and Visualization Libraries with pip
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2014/11/04/data-visualization-with-python
This snippet provides commands to install the necessary Python libraries, Pandas and Bokeh, using the `pip` package manager. These libraries are essential for data manipulation and visualization, respectively, and are prerequisites for the subsequent code examples.
```Bash
pip install pandas
pip install bokeh
```
--------------------------------
### Modulo Operator (%) Usage Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/modulo
Illustrates the behavior of the modulo operator (`%`) with numerical examples, showing how it calculates the remainder of a division. These examples demonstrate typical input-output pairs for the function.
```Conceptual
5 % 2
-- Result: "1"
4 % 2
-- Result: "0"
```
--------------------------------
### SoQL simplify Function Usage Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/simplify
Demonstrates practical applications of the `simplify` function with example geometries and different simplification levels, showing the resulting simplified GeoJSON output.
```SoQL
simplify(to_polygon('POLYGON ((0.0 0.0, 5.0 4.0, 11.0 5.5, 17.3 3.2, 27.8 0.1, 0.0 0.0))'), 5)
-- Result: {"type":"Polygon","coordinates":[[[0.0,0.0],[5.0,4.0],[11.0,5.5],[27.8,0.1],[0.0,0.0]]]}
simplify(to_polygon('POLYGON ((0.0 0.0, 5.0 4.0, 11.0 5.5, 17.3 3.2, 27.8 0.1, 0.0 0.0))'), 4)
-- Result: {"type":"Polygon","coordinates":[[[0.0,0.0],[11.0,5.5],[27.8,0.1],[0.0,0.0]]]}
```
--------------------------------
### SoQL Add Function Usage Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/add
Illustrates how to use the '+' function in SoQL for both unary (preserving sign) and binary (addition) numeric operations. Examples demonstrate different numeric inputs and their expected results.
```SoQL
Keep a number's sign
Add two numbers together
Examples:
+(2)
-- Result: "2"
+(-2)
-- Result: "-2"
2 + 2
-- Result: "4"
2.5 + 3
-- Result: "5.5"
```
--------------------------------
### FME Socrata Writer Log Output for New Dataset ID
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2014/10/09/fme-socrata-writer
This snippet shows an example of the log output from the FME Socrata Writer after successfully importing a new dataset. The log entry provides the newly generated dataset ID, which is essential for subsequent operations like updating the dataset.
```Log
Socrata Writer: 'TEMP.csv' was successfully imported as a new Socrata dataset. The dataset ID is 'k5ad-vnv2'
```
--------------------------------
### Install Ember CLI Chart.js Add-on
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/06/30/quickly-building-an-ember-app-backed-by-socrata-open-data
Installs the `ember-cli-chartjs` add-on, which wraps Chart.js, into an Ember.js project to enable chart components.
```Shell
$ ember install ember-cli-chartjs
```
--------------------------------
### Install Ember Socrata and Dependencies
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/06/30/quickly-building-an-ember-app-backed-by-socrata-open-data
Installs the `ember-socrata` add-on for Ember Data integration with Socrata, along with its required dependencies `ember-browserify` and `soda-js`.
```Shell
$ ember install ember-socrata
$ ember install ember-browserify
$ npm install --save-dev soda-js
```
--------------------------------
### Install RSocrata from GitHub
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2015/06/17/forecasting_with_rsocrata
Installs the RSocrata package directly from its GitHub repository, which is necessary for accessing the latest features not yet available on CRAN. This step requires the 'devtools' package.
```R
install_github("Chicago/RSocrata")
```
--------------------------------
### SoQL Multiply Function Usage Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/multiply
Demonstrates how to use the `*` (multiply) function in SoQL queries with concrete numerical examples, showing the input expressions and their corresponding results.
```SoQL
Multiply two numbers together
Examples:
2 * 2
-- Result: "4"
3 * 5
-- Result: "15"
```
--------------------------------
### Importing Pandas Library and Checking Version
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/files/pandas-and-jupyter-notebook
This snippet demonstrates how to import the pandas library, aliasing it as 'pd' for convenience. It also prints the installed pandas version to verify the setup and ensure the correct library is loaded.
```python
import pandas as pd
print(pd.__version__)
```
--------------------------------
### FME Workspace Command-Line Execution Example
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/data/excel2socrata_simple
Example command-line invocation for running the `excel2socrata_simple.fmw` FME workspace. It specifies the FME engine path, the workspace file, and parameters for the source Excel dataset (`--SourceDataset_XLSXR`) and the destination Socrata domain (`--DestDataset_SOCRATA`). This demonstrates how to automate FME workflows.
```Bash
"/Applications/FME 2014/Engine/fme" /tmp/wb_template_1414541529146_98251/excel2socrata_simple.fmw
--SourceDataset_XLSXR /Users/adrian/Dropbox/Socrata_s/projects/data_integration/fme_integration/fme-workflows/Demos/1/Downloads/Food_Inspections_small.xlsx
--DestDataset_SOCRATA socratau.demo.socrata.com
```
--------------------------------
### to_floating_timestamp Usage Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/to_floating_timestamp
Illustrative examples demonstrating how to use the `to_floating_timestamp` function with different input text and formatting strings, showing the expected output.
```APIDOC
to_floating_timestamp('4/1/2018 1:05:26AM','%m/%d/%Y %I:%M:%S%P')
-- Result: "2018-04-01T01:05:26"
```
--------------------------------
### Fetch Events from Socrata API using jQuery AJAX
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2017/03/30/creating-a-monthly-calendar-with-fullcalendar-io
Demonstrates making a GET request to a Socrata API endpoint using jQuery's `$.ajax()`. It filters events by date (last 31 days using Moment.js), city ('Portland'), and orders them by start time. The `.done()` callback is prepared for response handling.
```javascript
$(document).ready(function() {
// Fetch our events
$.ajax({
url: "https://data.oregon.gov/resource/yid5-c4eq.json",
method: "GET",
datatype: "json",
data: {
"$where" : "start_date_time > '" + moment().subtract(31, 'days').format("YYYY-MM-DDT00:00:00") + "'",
"city" : "Portland",
"$order" : "start_date_time DESC"
}
}).done(function(response) {
// TODO: Handle our response
});
});
```
--------------------------------
### SoQL to_number Function Usage Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/to_number
Demonstrates how to use the `to_number` function in SoQL to cast various values to a number. Shows examples with string literals and column references, illustrating the expected output.
```SoQL
cast a value to a number
Examples:
to_number('42')
-- Result: "42"
to_number(text_number_column)
-- Result: "42"
```
--------------------------------
### Get help for pandas.read_json method
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/02/01/pandas-and-jupyter-notebook
This snippet demonstrates how to access the documentation for a specific pandas method, `read_json`, directly within a Jupyter notebook. Appending a '?' to the method name displays its docstring, providing details on parameters, return types, and usage examples.
```Python
pd.read_json?
```
--------------------------------
### Example: Using http_get with json_pluck
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/http_get
An example demonstrating how to use the `http_get` function to fetch JSON data from a URL and then extract a specific value using `json_pluck`. This illustrates a common pattern for integrating external data sources within the Socrata query environment.
```SoQL
json_pluck(
http_get('https://some-service-returning/some-json.json'),
'.some.value'
)
```
--------------------------------
### left_pad Function Definition and Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/left_pad
Documents the `left_pad` function, explaining its purpose to pad a string to a desired length. It includes practical examples demonstrating its usage with different padding characters and lengths, along with its type signature.
```Conceptual
Pad `text` with the minimum number of copies of `pad` to reach `desired_length`.
Examples:
left_pad('hello', 10, 'x')
-- Result: "xxxxxhello"
left_pad('hello', 10, 'xy')
-- Result: "xyxyxhello"
```
```APIDOC
text, number, text -> text
```
--------------------------------
### HTTP Basic Authentication POST Request Example
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/authentication
An example HTTP POST request demonstrating the use of HTTP Basic Authentication and an X-App-Token for accessing a Socrata API endpoint. The Authorization header is typically generated by an HTTP library, not manually constructed.
```HTTP
POST /api/v3/views/4tka-6guv/query.json HTTP/1.1
Host: soda.demo.socrata.com
Accept: */*
Authorization: Basic [REDACTED]
Content-Length: 253
Content-Type: application/json
X-App-Token: [REDACTED]
[ {
...
} ]
```
--------------------------------
### Example Usage of location_to_point Function
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/location_to_point
Demonstrates how to use the `location_to_point` function to convert a location value into a point, showing an example with a `make_location` call and its resulting GeoJSON Point object.
```APIDOC
Turn a location value into a point
Examples:
location_to_point(make_location('3331 39th Ave', 'Vancouver', 'WA', '98818', to_point('POINT (42 42)')))
-- Result: {"type":"Point","coordinates":[42,42]}
```
--------------------------------
### SoQL Function Example: replace_first
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/replace_first
Demonstrates the usage of the `replace_first` function, showing how it replaces the first occurrence of a substring within a given text. The example illustrates replacing 'text' with an empty string in 'some-text-text'.
```SoQL
replace the first occurrence of a piece of text with another piece of text
Examples:
replace_first('some-text-text', 'text', '')
-- Result: "some--text"
```
--------------------------------
### to_fixed_timestamp Function Usage Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/to_fixed_timestamp
Practical examples demonstrating how to use the `to_fixed_timestamp` function with various date/time string inputs and corresponding formatting patterns to achieve desired datetime conversions.
```APIDOC
-- '%+' is shorthand for an ISO8601 compliant format
to_fixed_timestamp('2017-12-13T00:24:53.436562Z', '%+')
-- Result: "2017-12-13T00:24:53.436562+00:00"
-- Using %r shorthand for %H:%M:%S %p
to_fixed_timestamp('January 27, 2017 at 01:35:50 PM +0930', '%B %d, %Y at %r %z')
-- Result: "2017-01-27T04:05:50+00:00"
```
--------------------------------
### SoQL less_than Function Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/less_than
Demonstrates the behavior of the `less_than` function with various data types, showing how it evaluates numerical and string comparisons and returns a boolean result.
```SoQL
1 < 2
-- Result: true
1 < 1
-- Result: false
'a' < 'b'
-- Result: true
```
--------------------------------
### WKT Point Format Example
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/functions/intersects
An example of a Point encoded in Well-Known Text (WKT) format, demonstrating the 'longitude, latitude' coordinate order (X coordinate, Y coordinate).
```Text
POINT (-87.6174045 41.8669236)
```
--------------------------------
### Example Socrata API Response for Complaint Data
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/05/03/natural-language-with-sodapy-and-algorithmia
An example JSON response structure received from the Socrata API when querying for consumer complaints. It illustrates the typical fields and data types returned.
```JSON
[{'zip_code': '273XX', 'complaint_id': '1896417', 'issue': 'Disclosure verification of debt', 'company': 'CFS2, Inc.', 'submitted_via': 'Web', 'date_received': '2016-04-26T04:31:16', 'complaint_what_happened': 'I disputed a CFS2 trade-line on XXXX of my credit reports. These were initially disputed on XXXX XXXX, XXXX to XXXX via certified mail and XXXX on XXXX XXXX, XXXX. Each of these were verified by CFS2. \n\nI subsequently received letters from CFS2 dated XXXX XXXX, XXXX and XXXX XXXX, XXXX stating they did not have the necessary information to verify my account and would need 60-90 days to do this. I disputed this trade-line again with both XXXX ( XXXX XXXX, XXXX ) & XXXX ( XXXX XXXX, XXXX ) and CFS2 verified both accounts again. \n', 'sub_issue': 'Not given enough info to verify debt', 'state': 'NC', 'company_response': 'Closed with explanation', 'consumer_consent_provided': 'Consent provided', 'product': 'Debt collection', 'consumer_disputed': 'No', 'date_sent_to_company': '2016-04-26T04:31:17', 'timely': 'Yes', 'company_public_response': 'Company believes it acted appropriately as authorized by contract or law', 'sub_product': 'Credit card'}]
```
--------------------------------
### SoQL make_point Function Usage and Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/make_point
Demonstrates how to use the `make_point` function in SoQL to create a geographic point from latitude (Y) and longitude (X) coordinates. Includes examples showing both direct number inputs and conversion from text columns, along with their expected JSON output.
```SoQL
function to make a point out of a Y (latitude) and X (longitude) coordinate.
Note that the Y component (latitude) comes first
Examples:
make_point(47.123124, -123.325232)
-- Result: {"type":"Point","coordinates":[-123.325232,47.123124]}
make_point(to_number(text_number_column), to_number(text_number_column))
-- Result: {"type":"Point","coordinates":[42,42]}
```
--------------------------------
### SoQL date_extract_mm function examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/date_extract_mm
Examples demonstrating the usage of the `date_extract_mm` function with different timestamp inputs, including fixed and floating timestamps, and showing how timezone offsets can affect the extracted minute.
```SoQL
date_extract_mm(to_fixed_timestamp('2017-12-13T00:24:53Z'))
-- Result: "24"
date_extract_mm(to_floating_timestamp('2017-12-13T8:24:53'))
-- Result: "24"
date_extract_mm(to_fixed_timestamp('2017-12-13T00:24:53+0930'))
-- Result: "54"
```
--------------------------------
### GeoJSON MultiLineString Structure Example
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/datatypes/multiline
Illustrates the GeoJSON 'MultiLineString' format for the `MultiLine` datatype, showing how a set of paths is represented by arrays of WGS84 longitude and latitude coordinate pairs. This example demonstrates a basic two-line structure.
```JSON
{
"type": "MultiLineString",
"coordinates": [
[ [100.0, 0.0], [101.0, 1.0] ],
[ [102.0, 2.0], [103.0, 3.0] ]
]
}
```
--------------------------------
### Complete Socrata Data Visualization on Google Map Example
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2014/05/31/google-maps
This comprehensive JavaScript code integrates all the previous steps: constructing the Socrata API query, initializing the Google Map, fetching the data, and plotting markers. The entire logic is wrapped in '$(window).load' to ensure it runs after the DOM is fully loaded, providing a complete, runnable example for visualizing Socrata data on a Google Map.
```JavaScript
$(window).load(function() {
// Construct the query string
url = 'https://data.ct.gov/resource/v4tt-nt9n.json?'
+ 'organization_type=Public%20School%20Districts'
+ '&$$app_token=09sIcqEhoY0teGY5rhupZGqhW';
// Intialize our map
var center = new google.maps.LatLng(41.7656874,-72.680087);
var mapOptions = {
zoom: 8,
center: center
}
var map = new google.maps.Map(document.getElementById("map"), mapOptions);
// Retrieve our data and plot it
$.getJSON(url, function(data, textstatus) {
$.each(data, function(i, entry) {
var marker = new google.maps.Marker({
position: new google.maps.LatLng(entry.location_1.coordinates[1],
entry.location_1.coordinates[0]),
map: map,
title: location.name
});
});
});
});
```
--------------------------------
### Geocode ESRI Function API Reference and Usage Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/geocode_esri
Comprehensive documentation for the `geocode_esri` function, detailing its purpose, parameters, rate limiting, batching, field name customization, and usage examples for geocoding addresses with ESRI ArcREST.
```APIDOC
geocode_esri is a function which takes human readable addresses
and translates them into a latitude, longitude point using an ESRI
ArcREST server which is publicly accessible on the internet.
It's highly recommended that you specify a rate limit to avoid DDOSing
your ESRI ArcREST server. The last argument to this function is a rate limit
in requests per minute. It will default to 300, which means that no more than
300 geocode requests will be made in 1 minute. If your ArcREST instance is having
trouble keeping up with the request rate, consider lowering this value.
Each request to your ArcREST server contains a batch of 150 addresses (or the
server's SuggestedBatchSize if there is one), so if you have a rate limit of 10
requests per minute, it will take about 1 minute to geocode 1000 addresses.
Some ArcREST servers are set up to accept different field names than the defaults. The default
behavior is to POST a list of locations, which looks like
[
{"Address": "1111 Something AVE NW", "City": "New York", "Region": "New York", "Postal": "11111" }
]
However, some ArcREST servers instead call the "Address" component "Street", or the "Region" component "State", or the "Postal" component "ZIP", etc.
You can override these field names explicitly, using the final 4 arguments to the function. See the final example below.
There are several versions of the geocode_esri function:
geocode_esri(address: text, city: text, state: text, zip: text, esri_url: text)
This version defaults to geocoding within the United States
geocode_esri(address: text, city: text, state: text, zip: text, esri_url: text, rate_limit: number)
This version defaults to geocoding within the United States and applies a rate limit
geocode_esri(address: text, city: text, state: text, zip: text, country: text, esri_url: text)
This version allows you to specify which country
geocode_esri(address: text, city: text, state: text, zip: text, country: text, esri_url: text, rate_limit: number)
This version allows you to specify which country and applies a rate limit
geocode_esri(address: text, city: text, state: text, zip: text, country: text, esri_url: text, rate_limit: number, address_field_name: text, city_field_name: text, state_field_name: text, zip_field_name: text)
This version allows you to override the default location field names with your own, which are specific to your ArcREST server configuration
```
```Socrata Expression Language
Examples:
geocode_esri(`my_address_column`, 'Seattle', 'WA', `zipcode_column`, 'https://my-esri-arcrest-server.gov/arcgis/rest/services/Locators/name/GeocodeServer')
geocode_esri(`my_address_column`, 'Seattle', 'WA', `zipcode_column`, 'https://my-esri-arcrest-server.gov/arcgis/rest/services/Locators/name/GeocodeServer', 60)
geocode_esri(`my_address_column`, 'Seattle', 'WA', `zipcode_column`, 'US', 'https://my-esri-arcrest-server.gov/arcgis/rest/services/Locators/name/GeocodeServer')
geocode_esri(`my_address_column`, 'Seattle', 'WA', `zipcode_column`, 'US', 'https://my-esri-arcrest-server.gov/arcgis/rest/services/Locators/name/GeocodeServer', 20)
geocode_esri(`my_address_column`, 'Seattle', 'WA', `zipcode_column`, 'US', 'https://my-esri-arcrest-server.gov/arcgis/rest/services/Locators/name/GeocodeServer', 20, "Street", "City", "State", "ZIP")
```
--------------------------------
### Example Algorithmia Sentiment Analysis Result
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/05/03/natural-language-with-sodapy-and-algorithmia
An example response object from the Algorithmia Social Sentiment Analysis API, showing sentiment scores (neutral, positive, negative, compound) for the input sentence, along with metadata.
```Python Output
AlgoResponse(result=[{'neutral': 0.897, 'positive': 0.027, 'compound': -0.5574, 'sentence': 'I disputed a CFS2 trade-line on XXXX of my credit reports. These were initially disputed on XXXX XXXX, XXXX to XXXX via certified mail and XXXX on XXXX XXXX, XXXX. Each of these were verified by CFS2. \n\nI subsequently received letters from CFS2 dated XXXX XXXX, XXXX and XXXX XXXX, XXXX stating they did not have the necessary information to verify my account and would need 60-90 days to do this. I disputed this trade-line again with both XXXX ( XXXX XXXX, XXXX ) & XXXX ( XXXX XXXX, XXXX ) and CFS2 verified both accounts again. \n', 'negative': 0.076}], metadata=Metadata(content_type='json', duration=0.127483706, stdout=None))
```
--------------------------------
### Navigate and Serve Ember Application Locally
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/06/30/quickly-building-an-ember-app-backed-by-socrata-open-data
After initializing an Ember project, these commands allow you to change into the newly created project directory and then start a local development server. This enables testing the application in a web browser, typically accessible at http://localhost:4200.
```shell
cd cta-ridership
ember serve
```
--------------------------------
### Initialize a New Ember Project with Ember CLI
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/06/30/quickly-building-an-ember-app-backed-by-socrata-open-data
This command uses Ember CLI to create a new project directory, set up application scaffolding, install dependencies, and initialize a Git repository. It's the essential first step in setting up any new Ember application.
```shell
ember new cta-ridership
```
--------------------------------
### Socrata Developer Documentation Index
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2014/09/05/webinar-getting-started-civic-application-development
This section outlines the main topics covered in the Socrata developer documentation, providing a structured overview of API concepts, data querying, supported data formats, and various datatypes. It serves as a table of contents for the platform's capabilities.
```APIDOC
Core Concepts:
- Row Identifiers
- RESTful Verbs
- Application Tokens
- Authentication
- Response Codes & Headers
- System Fields
- CORS & JSONP
Querying:
- SoQL Queries
- SoQL Function and Keyword Listing
- Data Transform Functions
Data Formats:
- JSON
- GeoJSON
- CSV
- RDF-XML
Datatypes:
- Checkbox
- Fixed Timestamp
- Floating Timestamp
- Line
- Location
- MultiLine
- MultiPoint
- MultiPolygon
- Number
- Point
- Polygon
- Text
- URL
Other Resources:
- Other APIs
- Libraries & SDKs
```
--------------------------------
### HTML Setup for Highcharts Chart Container
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2015/03/30/create-a-column-chart-with-highcharts
This HTML snippet provides the essential structure for a web page integrating Highcharts. It includes script tags for jQuery and Highcharts libraries, along with a `div` element (`#chart-container`) designated as the target for the Highcharts visualization. This setup ensures all necessary dependencies are loaded and a rendering area is available.
```html
```
--------------------------------
### APIDOC: `slice` Function Usage and Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/slice
Documents the `slice` function, which extracts a substring of a specified length from a text starting at a given index. Includes various examples demonstrating its behavior with different inputs, including valid and invalid indices and lengths.
```SoQL
Get a substring of a specified length of a text from a start index
Examples:
slice('foobar', 0, 2)
-- Result: "fo"
slice('foobar', -3, 3)
-- Result: "bar"
slice('foobar', -10, 1300)
-- Result: ""
slice('foobar', 0, -2)
-- Result: {"type":"invalid_length","english":"Invalid length -2. Slice length must be a nonnegative integer.","data":{"length":-2}}
```
```APIDOC
text, number, number -> text
```
--------------------------------
### SoQL to_multipoint Function Usage and Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/to_multipoint
This snippet provides the functional description of the `to_multipoint` function in SoQL, explaining its ability to convert point or WKT text representations into a multipoint value. It includes practical examples demonstrating how to use the function with different input types and the expected JSON output format.
```SoQL Examples
convert a point into a multipoint
parse a WKT (text) representation of a multipoint into a multipoint value
Examples:
to_multipoint(to_point('POINT (1 2)'))
-- Result: {"type":"MultiPoint","coordinates":[[1,2]]}
to_multipoint('MULTIPOINT ((10 40), (40 30), (20 20), (30 10))')
-- Result: {"type":"MultiPoint","coordinates":[[10,40],[40,30],[20,20],[30,10]]}
to_multipoint(a_wkt_multipoint)
-- Result: {"type":"MultiPoint","coordinates":[[10,40],[40,30],[20,20],[30,10]]}
```
```APIDOC
Signatures:
point -> multipoint
text -> multipoint
multipoint -> multipoint
```
--------------------------------
### Make a JSONP GET Request to Socrata API with jQuery
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/cors-and-jsonp
This jQuery AJAX example illustrates how to perform a cross-origin GET request using JSONP, a technique for older browsers or specific scenarios where CORS is not supported. It specifies the `$jsonp` parameter for the callback function and uses the `dataType: 'jsonp'` setting. The `.done()` callback handles the successful response.
```JavaScript
$.ajax({
url: "https://data.chattlibrary.org/resource/e968-fnk9.json",
jsonp: "$jsonp",
dataType: "jsonp"
}).done(function(data) {
console.log("Request received: " + data);
});
```
--------------------------------
### Socrata Platform API Concepts and Data Formats
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2015/05/20/data-visualisation-with-r
This section outlines key concepts and components of the Socrata platform's API, including data access methods, supported data formats, and various datatypes. It serves as a high-level reference for interacting with Socrata datasets.
```APIDOC
Row Identifiers
RESTful Verbs
Application Tokens
Authentication
Response Codes & Headers
System Fields
CORS & JSONP
Querying:
SoQL Queries
SoQL Function and Keyword Listing
Data Transform Functions
Data Formats:
JSON
GeoJSON
CSV
RDF-XML
Datatypes:
Checkbox
Fixed Timestamp
Floating Timestamp
Line
Location
MultiLine
MultiPoint
MultiPolygon
Number
Point
Polygon
Text
URL
Other APIs
Libraries & SDKs
```
--------------------------------
### Import pandas and check version
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/02/01/pandas-and-jupyter-notebook
This snippet imports the pandas library, aliasing it as 'pd' for convenience. It then prints the installed version of pandas to verify the environment setup, ensuring the correct library version is in use.
```Python
import pandas as pd
print(pd.__version__)
> 0.17.1
```
--------------------------------
### Socrata API Documentation Structure Overview
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2017/03/30/creating-a-monthly-calendar-with-fullcalendar-io
This snippet outlines the main sections and topics covered in the Socrata API documentation, providing a high-level view of the available resources, including core API concepts, querying capabilities, supported data formats, and various datatypes.
```APIDOC
Socrata API Documentation Topics:
- Core Concepts:
- Row Identifiers
- RESTful Verbs
- Application Tokens
- Authentication
- Response Codes & Headers
- System Fields
- CORS & JSONP
- Querying:
- SoQL Queries
- SoQL Function and Keyword Listing
- Data Transform Functions
- Data Formats:
- JSON
- GeoJSON
- CSV
- RDF-XML
- Datatypes:
- Checkbox
- Fixed Timestamp
- Floating Timestamp
- Line
- Location
- MultiLine
- MultiPoint
- MultiPolygon
- Number
- Point
- Polygon
- Text
- URL
- Other APIs
- Libraries & SDKs
```
--------------------------------
### polylabel Function API Reference and Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/polylabel
API documentation for the `polylabel` function, which calculates a point guaranteed to be within a polygon's boundaries using a recursive grid-based algorithm. This entry includes the function's detailed description, practical examples demonstrating its use with both single and multi-polygons, and its formal signature.
```APIDOC
Returns a point that must exist within the polygon borders. It uses the recursive grid-based algorithm described here: https://github.com/mapbox/polylabel#how-the-algorithm-works. When given a multipolygon, the point it returns is within the largest (by area) sub-polygon.
Examples:
polylabel(to_polygon('POLYGON ((0.0 0.0, 4.0 0.0, 4.0 1.0, 1.0 1.0, 1.0 4.0, 0.0 4.0, 0.0 0.0))'))
-- Result: {"type":"Point","coordinates":[0.5859375,0.5859375]}
polylabel(to_multipolygon('MULTIPOLYGON (((0.0 0.0, 4.0 0.0, 4.0 1.0, 1.0 1.0, 1.0 4.0, 0.0 4.0, 0.0 0.0)),((6.0 6.0, 7.0 6.0, 7.0 7.0, 6.0 7.0, 6.0 6.0)))')) -- a big L-shape and a little square
-- Result: {"type":"Point","coordinates":[0.5859375,0.5859375]}
Signatures:
p -> point
```
--------------------------------
### APIDOC: SoQL `trim` Function Definition and Usage
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/trim
Documentation for the `trim` function, which removes specified characters from the start and end of a string. It takes two arguments: the string to be trimmed and the character(s) to remove. The examples illustrate its behavior with various inputs, and its type signature is provided.
```APIDOC
Function: `trim`
trim characters off the start and end of a string
Examples:
trim(' SOMETHING ', ' ')
-- Result: "SOMETHING"
trim(',a,b,c,d,', ',')
-- Result: "a,b,c,d"
trim(' SOMETHING ', 'nope')
-- Result: " SOMETHING "
Signatures:
text, text -> text
```
--------------------------------
### Initialize Socrata Client for Data Access
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/05/03/natural-language-with-sodapy-and-algorithmia
Initializes the Socrata client for API interaction, requiring a domain, an application token, username, and password for authentication.
```Python
from sodapy import Socrata
socrataClient = Socrata("data.consumerfinance.gov", "myAppToken",
username="user.name@domain.com",
password="passw0rd!")
```
--------------------------------
### Example Usage of Deprecated make_location Function (SoQL)
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/make_location
This function has been deprecated. Please use the `make_point` function instead. `make_location` makes a location column from human readable address columns and a point column. A Location column is the amalgamation of these two components. This example demonstrates how to use `make_location` with address parts and a generated point.
```SoQL
make_location(
`my_address_column`,
'Seattle',
'WA',
'98118',
make_point(
to_number(`my_latitude_column`),
to_number(`my_longitude_column`)
)
)
```
--------------------------------
### SoQL Function: title_case
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/title_case
Documents the `title_case` function available in SoQL (Socrata Query Language), which converts a string to title case. It specifies exceptions for small words as defined by the NYT Style Guide and provides usage examples with expected results, along with its type signature.
```APIDOC
Make string title case with the exception of small words as defined by NYT Style Guide:
"a","an","and","as","at","but","by","en","for","if","in","of","on","or","the","to","viz","vs","vs."
Examples:
title_case('department of transportation')
-- Result: "Department of Transportation"
title_case('a very interesting story')
-- Result: "A Very Interesting Story"
title_case('First name, last name')
-- Result: "First Name, Last Name"
text -> text
```
--------------------------------
### Example RDF-XML Data Request URL
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/formats/rdf-xml
This URL demonstrates how to request a single record from a Socrata dataset in RDF-XML format. The `$limit=1` parameter restricts the response to one entry.
```URL
https://data.cityofchicago.org/resource/xzkq-xp2w.rdf?$limit=1
```
--------------------------------
### Parse URI with `uri_parse` function examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/uri_parse
Demonstrates the usage of the `uri_parse` function to extract components from a URI string. It shows how to access scheme, host, path, TLD, and query, and illustrates its behavior with both valid and invalid URLs.
```SoQL
uri_parse('https://socrata.com/hello').scheme
-- Result: "https"
uri_parse('https://socrata.com/hello').host
-- Result: "socrata.com"
uri_parse('https://socrata.com/hello').path
-- Result: "/hello"
uri_parse('https://socrata.com/hello').tld
-- Result: "COM"
uri_parse('https://socrata.com/hello?foo=bar').query
-- Result: "foo=bar"
uri_parse('not a url').scheme
-- Result: null
uri_parse('not a url').host
-- Result: null
uri_parse('not a url').path
-- Result: "not a url"
uri_parse('not a url').tld
-- Result: null
uri_parse('not a url').query
-- Result: null
```
--------------------------------
### Socrata Client `get` Method Parameters
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/05/03/natural-language-with-sodapy-and-algorithmia
Documentation for the `socrataClient.get()` method in `sodapy`, outlining common parameters for querying Socrata datasets, including filtering and limiting options.
```APIDOC
Method: socrataClient.get(dataset_id, content_type="json", limit=None, where=None, **kwargs)
Parameters:
dataset_id (string): The unique identifier for the Socrata dataset (e.g., "nsyy-je5y").
content_type (string, optional): Desired format of the response. Defaults to "json".
limit (int, optional): Maximum number of records to return.
where (string, optional): SoQL WHERE clause for filtering records (e.g., "date_received > '2015-01-01'").
**kwargs: Additional key-value pairs for direct column filtering (e.g., company="Equifax").
Return Type: List of dictionaries (JSON objects) representing the queried records.
Purpose: Retrieves data from a specified Socrata dataset with various filtering and limiting options.
```
--------------------------------
### Query Socrata API with PowerShell Invoke-RestMethod
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/08/24/powershell-and-soda
This PowerShell command demonstrates how to retrieve data from a Socrata Open Data API endpoint using the `Invoke-RestMethod` cmdlet. It performs a GET request to a specified JSON resource URL. This method is suitable for simple data retrieval from RESTful services and requires PowerShell to be installed.
```PowerShell
Invoke-RestMethod -Uri https://data.oregon.gov/resource/uq5y-tk6r.json -Method get
```
--------------------------------
### Calculate Column-Specific Statistics and Unique Values in Pandas
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2016/02/01/pandas-and-jupyter-notebook
This section demonstrates several pandas methods for obtaining specific statistics and unique value information from individual DataFrame columns. It includes examples for calculating the mean (`.mean()`), counting non-null values (`.count()`), retrieving all unique values (`.unique()`), and getting the frequency distribution of unique values (`.value_counts()`).
```python
df.bachelor_s_degree_or_higher.mean()
df.geography.count()
df.geography_type.unique()
df.less_than_high_school_graduate.value_counts()
```
--------------------------------
### SoQL Functions Reference
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/functions
Lists all available functions in SoQL with their descriptions, parameters, and API version availability.
```APIDOC
Function: avg(...)
Description: Returns the average of a given set of numbers
Availability: 2.0 and 2.1
Function: between ... and ...
Description: Returns TRUE for values in a given range
Availability: 2.1
Function: case(...)
Description: Returns different values based on the evaluation of boolean comparisons
Availability: 2.1
Function: convex_hull(...)
Description: Returns the minimum convex geometry that encloses all of another geometry
Availability: 2.1
Function: count(...)
Description: Returns a count of a given set of records
Availability: 2.0 and 2.1
Function: date_extract_d(...)
Description: Extracts the day from the date as an integer.
Availability: 2.1
Function: date_extract_dow(...)
Description: Extracts the day of the week as an integer between 0 and 6 (inclusive).
Availability: 2.1
Function: date_extract_hh(...)
Description: Extracts the hour of the day as an integer between 0 and 23 (inclusive).
Availability: 2.1
Function: date_extract_m(...)
Description: Extracts the month as an integer.
Availability: 2.1
Function: date_extract_mm(...)
Description: Extracts the minute from the time as an integer.
Availability: 2.1
Function: date_extract_ss(...)
Description: Extracts the second from the time as an integer.
Availability: 2.1
Function: date_extract_woy(...)
Description: Extracts the week of the year as an integer between 0 and 51 (inclusive).
Availability: 2.1
Function: date_extract_y(...)
Description: Extracts the year as an integer.
Availability: 2.1
Function: date_trunc_y(...)
Description: Truncates a calendar date at the year threshold
Availability: 2.0 and 2.1
Function: date_trunc_ym(...)
Description: Truncates a calendar date at the year/month threshold
Availability: 2.0 and 2.1
Function: date_trunc_ymd(...)
Description: Truncates a calendar date at the year/month/date threshold
Availability: 2.0 and 2.1
Function: distance_in_meters(...)
Description: Returns the distance between two Points in meters
Availability: 2.1
Function: extent(...)
Description: Returns a bounding box that encloses a set of geometries
Availability: 2.1
Function: greatest(...)
Description: Returns the largest value among its arguments, ignoring NULLs.
Availability: 2.1
Function: in(...)
Description: Matches values in a given set of options
Availability: 2.1
Function: intersects(...)
Description: Allows you to compare two geospatial types to see if they intersect or overlap each other
Availability: 2.1
Function: least(...)
Description: Returns the smallest value among its arguments, ignoring NULLs.
Availability: 2.1
Function: like '...'
Description: Allows for substring searches in text strings
Availability: 2.1
Function: ln(...)
Description: Returns the natural log of a number
Availability: 2.1
Function: lower(...)
Description: Returns the lowercase equivalent of a string of text
Availability: 2.1
```
--------------------------------
### Ruby: Print 'Hello Data!'
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/index
A simple Ruby code snippet demonstrating how to print a string to the console using the `puts` function.
```Ruby
# puts "Hello Data!";
```
--------------------------------
### SoQL `region_code` Function Usage and Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/region_code
Demonstrates the usage of the `region_code` function in SoQL to identify the region a given point falls within. It shows how to combine `make_point` with `region_code` and provides examples of successful execution, permission errors, and no-match scenarios, along with a query example for grouping by region.
```APIDOC
Turn a point into the ID of a region, based on which region the point falls within. For example, if this dataset can produce
a Point column, and we have another dataset with ID 'poly-data' with Polygons or MultiPolygons, where
each Polygon represents a zip code, and it has a column with the zip code called 'zip', we could use the following expression
region_code(
make_point(
to_number(latitude),
to_number(longitude)
),
'poly-data',
'zip'
)
To create a column where the values are the zip codes that each point falls within.
Then, once we publish this dataset, we can easily figure out the number of rows that
fall within each region, by writing the following query
select count(*), `zip` group by `zip`
Examples:
region_code(make_point(2, 3), 'hehe-hehe')
-- Result: "7"
region_code(make_point(2, 3), 'forbidden-view')
-- Result: {"type":"region_coding_error","english":"Failed to Region Code that value; You don't have permission to region code against that dataset","data":{"message":"You don't have permission to region code against that dataset"}}
region_code(make_point(2, 3), 'no-match')
-- Result: {"type":"empty_region_code","english":"Region Coder said that point does not fall within any region","data":{}}
```
--------------------------------
### GeoJSON Point Representation Example
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/datatypes/point
An example of the Socrata Point datatype encoded as a GeoJSON 'Point' object, showing its type and coordinates. Note the 'longitude, latitude' ordering.
```JSON
{
"type": "Point",
"coordinates": [
-87.653274,
41.936172
]
}
```
--------------------------------
### Accessing Help Documentation for Pandas Functions
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/files/pandas-and-jupyter-notebook
This example shows how to quickly access the built-in help documentation for any pandas function within a Jupyter Notebook by appending a question mark to the function name. This is useful for understanding parameters, usage, and return types without leaving the notebook environment.
```python
pd.read_json?
```
--------------------------------
### SoQL Keywords Reference
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/functions
Lists all available keywords in SoQL with their descriptions and API version availability.
```APIDOC
Keyword: distinct
Description: Returns distinct set of records
Availability: 2.1
```
--------------------------------
### SoQL `not` Function Examples
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/transforms/not
Demonstrates the usage of the `not` function in SoQL to invert a boolean value. The examples show how `not true` evaluates to `false` and `not false` evaluates to `true`.
```SoQL
not true
-- Result: false
not false
-- Result: true
```
--------------------------------
### GeoJSON MultiPoint Structure Example
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/docs/datatypes/multipoint
Illustrates the basic GeoJSON structure for a MultiPoint datatype, showing its 'type' and 'coordinates' properties. Note that GeoJSON specifies coordinates as longitude, latitude (X, Y).
```JSON
{
"type": "MultiPoint",
"coordinates": [ [100.0, 0.0], [101.0, 1.0] ]
}
```
--------------------------------
### Load R Packages for Socrata Data and Visualization
Source: https://dev.socrata.com/foundry/data.cityofnewyork.us/ipu4-2q9a/blog/2015/05/20/data-visualisation-with-r
This R code uses the 'pacman' package to load several essential libraries: 'RSocrata' for Socrata API interaction, 'plyr' for data manipulation, and 'RColorBrewer' for color palettes in visualizations. These packages are crucial for fetching, processing, and visualizing data from the Socrata platform.
```R
p_load(RSocrata,plyr,RColorBrewer)
```