ArcGIS II

Before you begin

Outside of workshop: general GIS resources at MIT are available at http://libraries.mit.edu/gis.

1. You will need an MIT Geodata Repository Account before beginning this exercise
If you do not already have an account you can create one at: http://libraries.mit.edu/gis/data/repository.html

2. Open the windows command prompt

    1. type: attach –DT gis

navigate to T:\workspace2\workshops\arcgis2_iap07 and copy the contents of the “arcgis2” folder to your f:\usertemp\arcgis1

 

Introduction

 

This workshop will lead you through a variety of tools available in ArcGIS 9.1 and is geared toward the user who is already familiar with the basic functionality of ArcGIS (or has taken ArcGIS 1).  This exercise is set up for educational use only.  The data is not to be used for commercial purposes.  In this exercise you will learn to:

  1. Georeference - Align geographic data to a known coordinate system so it can be viewed, queried, and analyzed with other geographic data. Georeferencing may involve shifting, rotating, scaling, skewing, and in some cases warping or rubber sheeting an image.
  2. Adding tabular data
    1. Geocoding - associating a table of address data with a gis street file to create a point gis layer. You will learn to create a new address locator in ArcCatalog and Geocode a table of addresses in ArcMap.
    2. Adding xy data
  3. change the projection of a data layer
  4. calculate the distance between points in a shapefile
  5. Spatial Join, Dissolve, and Field Calculation
  6. create contour lines from a digital elevation model
  7. create and edit shapefiles
  8. Create a 3D model in ArcScene
    1. extrude buildings to display in 3D
    2. set a DEM as the base layer and drape other layers over top of it
    3. vertical exaggeration
    4. adjusting the image rendering

The Scenario:

Someone has approached you to help them choose a neighborhood/planning district in Boston to build an educational theater (ET).  Once the location is decided upon they would like a 3D model of the site.  The neighborhood chosen should have a high density of schools, be accessible by the T, so the kids can easily get to it, preferably not have other movie theaters nearby to compete with, and it should have open space nearby for the kids to try experiments after learning about new ideas at the ET.

 

Georeferencing a map

 

Add the gis file you will use to georeference the image

You found a paper map of the City of Boston Planning districts and decided to scan it and save it as a .jpg image so you could work with it in ArcGIS and analyze other layers with it.

First you will add a gis file that represents the City of Boston, so you will have something to georeference your map to.

Do an advanced search in the MIT Geodata Repository on keyword = “Boston” and keyword = “Border”

 

 

  1. Add the Border for the City of Boston to your ArcMap project (ma_boston_f7brdr_2001) from the MIT Geodata Repository
  2. Right click sde_data.us_ma_boston_f7brdr_2001 ->select properties-> select the general tab
  3. Change the layer name to “border”

 

Add and explore the image you will georeference

  1. Add F:\usertemp\arcgis2\bostonplanning.jpg to your ArcMap project
  2. ArcGIS may ask you if you want to build pyramids.  Click Yes (A pyramid file will enable ArcGIS to redraw your raster more quickly. This is more important for large raster files than small ones like this file.)
  3. You may get a warning about the “Unknown Spatial Reference: The following data sources you added are missing spatial reference information.  This data can be drawn in ArcMap, but cannot be projected.”  Click ok.  (Your next step will be to give this image spatial reference information.)
  4. The bostonplanning.jpg file appears as a layer in your Table of Contents but does not draw on top of your Boston borders file.
  5. Run your mouse over the Boston border file and notice the coordinates in the lower right hand corner of your project.
  6. Right click bostonplanning.jpg and select ‘zoom to layer’ then again examine the coordinates in the lower right hand corner of your project – the file is ‘out of this world’ and needs geographic coordinates assigned to make it appear in it’s appropriate geographic location.
  7. Right click the border file and select ‘zoom to layer’. 
  8. Change the symbology of your Boston border file to hollow center, red boundary with a width of 1 (so it will be easily distinguishable from your Boston planning map)

 

Georeference the Boston Planning map using the Boston borders file

  1. Turn on the georeferencing toolbar (right click in the toolbar area and select “georeferencing”)
  2. Make sure the layer on the georefencing toolbar is set to bostonplanning.jpg
  3. Click Georeferencing -> Fit to Display
  4. The Boston planning map will appear centered in your data frame.
  5. To georeference one could enter the coordinates of the map (if they are known) or one could assign coordinates to the scanned map by clicking on the scanned map then clicking in the same place on the gis file, which is what we will do in this exercise.
  6. Click ‘add control points’ on the georeferencing toolbar

 

 

  1. Find 3 places you can ensure are the same on the scanned map and the gis file and line them up by first clicking on the scanned map then clicking on the gis file.  The three locations chosen should be well distributed places around the map.  It can be helpful to zoom in to ensure the areas clicked are the same.
  2. Once your Boston planning map is well lined up with your Boston border gis file we will save it as a .TIF image so the coordinate information will be permanently stored in the header of the TIF file.  (This is what is known as a ‘geotiff’)
  3. click Georefencing -> rectify
  4. Save Rectifybostonplanning.tif to your F:\usertemp\arcgis2\ folder
  5. Add Rectifybostonplanning.tif to your project and make sure it looks ok

 

To convert this raster to a vector file you could use the extension ‘ArcScan’.  ArcScan has limited functionality and is very particular about how the files are set up so we will not get into it in this workshop.  More information is available in the ESRI Desktop Help.

 

After you georeferenced your map you discovered there is a BRA planning districts file that shows the same boundaries and is in vector format on the MIT Geodata Repository, so you don’t need to digitize it by hand (which can be very time consuming).

 

Clear your ArcMap project and turn off the georeferencing toolbar.

 

Mapping tabular data in a GIS

 

You found a list of movie theaters in the boston area and their addresses on the internet.  You then saved the addresses as a MS Access database (because this format works more reliably than .txt and .dbf in ArcGIS).  You are going to map them by geocoding.

 

Geocoding

Geocoding lets you associate a table of addresses with a GIS street file to create point locations on a map.

Create a new Address Locator file in ArcCatalog:

  1. Open ArcCatalog
  2. Click “Create a new Address Locator”

 

 

    1. Choose US Streets (File)

                                                              i.      (File is for shapefile, GDB would be for geodatabase)

    1. Name the Address Locator ‘boston’
    2. In the Primary Table: Reference Data: navigate to f:\usertemp\arcgis2\boston_streets.shp. 

                                                              i.      The software will fill in the rest of the fields for you with this file because this file’s source is ESRI’s streetmap USA.  If you are using another file, such as free streets available from the US Census Bureau,  you may need to associate the appropriate field names from the file with the appropriate section of the Address Locator. 

 

 

Note: StreetMap USA contains detailed streets for the entire US and has Address Locators already created.  StreetMap USA is available on disk as part of the ESRI Data and Maps Dataset in the GIS Lab: http://libraries.mit.edu/gis/data/findingaids/index.html#esri (It is not currently available on the MIT Geodata Repository due to its very large size.) For more sources of street information check out the transportation page on the MIT GIS Lab website: http://libraries.mit.edu/gis/data/themes/transportation.html

    1. Click ok
  1. Right click the “boston” address locator you just created and select copy
  2. Navigate to the f:/usertemp/arcgis2 folder and right click to paste the address locator in your folder, so it will be really easy to find later.
  3. close ArcCatalog

 

Geocode in ArcMap:

In ArcMap

  1. Add f:\usertemp\arcgis2\boston_streets.shp to your project
  2. Add f:\usertemp\arcgis2\movies.mdb -> double click and select the movies table
  3. right click the movies table to open it and note there is a field named ‘name’ and a field named ‘address’
  4. On the top menu bar in ArcMap click Tools -> geocoding -> geocode addresses
  5. In the “choose an address locator” dialog box: click add -> navigate to your folder -> select the boston address locator

  1. click “add” then click “ok”
  2. In the Geocode Addresses dialog
    1. Address table: movies
    2. Address input fields: address
    3. Save as type: shapefile, in the folder: f:\usertemp\arcgis2\movies_geo.shp
    4. Click ok

Note: the geocoding options let you adjust things like how similar a record must be to an address for it to be called a match

  1. Note that 3 addresses matched with a score of 80-100 and 1 matched with a score less than 80.  Check which one had a lower score by choosing the radio button marked ‘addresses with score < 80’ in the rematch criteria. Then click ‘match interactively’

 

 

  1. The Simons Imax does not have an address number on the central wharf.  Since ArcGIS wasn’t sure which of the 2 address ranges to match to, it chose one of the two addresses that tied and gave it a lower score matching score. One could modify the address in through this dialog box to look for other matches.  You could move this “Interactive Review” dialog box to the side and click on the different addresses to see where they are highlighted on the map.  They are right next to each other, so for this exercise we will accept the chosen address record.

  1. Click close then click done

 

Mapping x, y data

You found a file of schools and their locations listed by latitude and longitude that you now need to map.

 

Your x, y data can be latitude and longitude or another coordinate system – make sure you define it correctly.

 

  1. Add the file b_schools_xy.dbf to your ArcMap project. 
  2. Open the table (right click the file name -> open)
  3. Look at the coordinates and notice they are in Latitude, Longitude.
  4. Click Tools -> add x,y data
  5. Make sure the table you are reading is b_schools_xy.dbf . Use the field named longitude for your x data and latitude for your y data.
  6. To define your spatial reference click Edit-> Select -> Geographic Coordinate Systems -> World -> WGS 1984.prj
    1. This type of information should typically be included in the metadata (descriptive information) for a file. If you are working with data from a GPS unit WGS84 is the most commonly used spatial reference system.

 

 

  1. Click ok to create b_schools_xy Events
  2. to make this layer permanent export it as a new shapefile (right click b_schools_xy Events -> data -> export data
  3. save the file as f:\usertemp\arcgis2\boston_schools.shp

Note: a common error is to switch the x and y coordinates or leave out a – where necessary.  These errors will map your data in the wrong part of the world. X,Y data must be in decimal degrees (decimal degrees = degrees + minutes/60 + seconds/3600)

 

Project your movie theater data

The file of movie theater locations you geocoded is unprojected and set to Geographic Coordinate System (GCS), North American Datum 1983.  We can check this by right clicking the file name and selecting the source tab, where it displays the data source information.

 

 

We want to project this file to NAD_1983_StatePlane_Massachusetts_Mainland_FIPS_2001 (one of the most common projections used for the Boston area), so that our next step of calculating the distance between the movie theaters will be more accurate and so the units will output in meters in instead of decimal degrees.

 

1. Open ArcToolbox

 

 

2. Within ArcToolbox navigate to Data Management Tools -> Projections and Transformations -> Feature -> Project then double click

 

 

3. Set the “Project” dialog box to match the picture below:

In the “Output Coordinate System” box navigate to: Projected Coordinate Systems -> State Plane -> NAD 1983 -> NAD_1983_StatePlane_Massachusetts_Mainland_FIPS_2001

Unfortunately ESRI was not very explicit - the NAD 1983 will set the units to meters, NAD 1983 will set the units to feet.

 

Notice that I made my output file name include an abbreviated version of the projection I used so it will be easy to keep track of later.

  1. Click ok
  2. Upon completion, click close
  3. Repeat this process to project the school data to match the movie file.

 

Calculate the distance between all the schools and all the movie theaters

In ArcToolbox

  1. Navigate to Analysis Tools -> Proximity -> point distance

 

 

  1. Set the “input features” to: movies_geo_spnad83_meters
  2. set the “near features” to the school file
  3. Set the “output table” to: F:\usertemp\arcgis2_iap07\movie_school_distance.dbf
  4. Click ok
  5. When completed click close
  6. Switch to the “source” tab at the bottom of the table of contents to see movie_school_distance.dbf
  7. Right click movie_school_distance.dbf to open and explore it

Note the column names: the “INPUT_FID” column records match the unique FID records of the input file, which was movie theaters.  The “NEAR_FID” column matches the unique FID records of, which was schools.  The distance column is in the units of measure of the files (it is best to set all files to the same unit of measure before running this tool, so there is no confusion.)

 

Which schools and movie theaters are closest to each other?

 

Note: If you want to calculate the distance between two points along a road (instead of the straight line distance we just calculated), then you would use the Network Analyst Extension.  Learn more about using Network Analyst in the IAP session later this week: http://libraries.mit.edu/gis/teach/iap2007.html

 

Spatial Join and Dissolve to determine the density of schools per neighborhood

We want to choose the planning district that has the highest density of schools.  When we open the planning districts file we see that area is already calculated for us.  We open the file properties and check the source tab to see what units the file is in.  We find that it is in meters.  (The unit of measurement for length, area and perimeter will be the same as the units of the file at the time the calculation is done.  If the units of the file are changed the area, length or perimeter will not automatically change – they would need to be re-calculated.)

 

From the MIT Geodata Repository add the BRA planning districts

Basic search: boston

Select the BRA planning districts (us_ma_boston_g45plnng_2000) and it to your ArcMap project

 

We want to find out how many schools are in each neighborhood and will use the spatial join tool to do this.

  1. Right click sde_data.us_ma_boston_g45plnng_2000
  2. Choose joins and relates -> join
  3. Choose ‘join data from another layer based on spatial location’
  4. Join with boston_schools
  5. And choose the 2nd option – Each polygon will be given a summary…
  6. Name the file f:\usertemp\arcgis2\plan_school.shp

 

We only want to look at one record per neighborhood, so we will dissolve the file, based on neighborhood name.

  1. ArcToolbox -> Data Management Tools -> Generalization -> Dissolve
  2. Input features: plan_school
  3. Output feature class: plan_school_Dissolve.shp
  4. Dissolve Fields (this will be the field the file will be dissolved by)

name

Statistics Field: count

Statistics type: sum

Statistics Field: us_ma_bost (this is the area but the field name got cut when we did the spatial join)

Statistics type: sum

(note: any field you want to summarize must be numeric)

 

Calculate the density of schools

  1. Open the attribute table of plan_school_Dissolve.shp
  2. Click options -> add field
  3. Use the field name school_den (since there is a 10 character limit for field names in ArcGIS)
  4. Type: double
  5. Right click the field name school_den to calculate values
  6. In the field calculator enter the formula: [SUM_Count_]/ [SUM_us_ma_] *1000000
  7. Right click the field name school_den and sort

 

This elevation file is available in the GIS lab as part of the “Boston area lidar data from MASSGIS”.  More information about this file and other elevation data sources can be found at: http://libraries.mit.edu/gis/data/themes/elevation.html

 

You then do another search in the MIT geodata repository on ‘boston’ and find an ‘open space’ file and an ‘MBTA lines’ file.  You see Back Bay is serviced by the MBTA green line and notice the Boston Common, Public Garden, Charles River Reservation and Commonwealth Avenue Mall provide a lot of open space in the area.  There is no existing movie theater in Back Bay and it meets the other criteria listed earlier: the neighborhood chosen should have a high density of schools, be accessible by the T, so the kids can easily get to it, preferably not have other movie theaters nearby to compete with, and it should have open space nearby for the kids to try experiments after learning about new ideas at the ET.

 

Create contour lines to help choose an area that isn’t too steep for your new building

  1. Add F:/usertemp/ArcGIS2/bare237898.tif to ArcMap
  2. In the main ArcMap toolbar area right click and turn on the 3D analyst extension
  3. In the main ArcMap toolbar area click Tools -> Extensions and make sure there is a check mark next to 3D analyst (if the extension is not activate then the tools will be greyed out and inaccessible)
  4. On the 3D Analyst Toolbar Select: 3D Analyst -> Surface Analysis -> Contour
  5. Change the contour interval from 1 (meter) to 5 (meters) (The way we know the units of the intervals is because we figured out in advance that this Digital Elevation Model is in meters).
  6. Set the “output features” to create a new file in your F:/usertemp/arcgis2 folder.  (When naming contour line files it is helpful for me to include the intervals in the file name, so this file I would name “cont_5m.shp”)
  7. The areas that are steeper are the areas that have contour lines closer together. The Back Bay area is not steep, so we are not concerned about building our new site on a steep slope.

Note: The Digital Elevation model session offered later today will go much more in depth about using DEMs.  More information is available at:: http://libraries.mit.edu/gis/teach/iap2007.html

 

We are interested in existing buildings outlines for Back Bay, for our 3D model. Do another search in the MIT Geodata Repository on boston and add the “Boston Metro, MA (Building Footprints and heights from LIDAR, 2002).   Select the Back Bay/ Beacon Hill polygon, and do a “select by location” the same way you did in the ArcGIS 1 exercise: http://libraries.mit.edu/gis/teach/iap07/arcgis1_iap07.html. Once you have selected the building footprints for Back Bay/ Beacon Hill right click the file name -> click data -> export data, and choose only the “selected features” for export to the new dataset. Name the new file “bldgs_bb.shp”.

 

 

Creating and Editing a new shapefile

You are going to create a new building for your site, in shapefile format.

Create a new shapefile in ArcCatalog

1. In ArcCatalog

2. Navigate to F:\usertemp\arcgis2\ -> right click in the folder -> select “new” -> “shapefile”

3. Set your shapefile to match the information below:

            Name: site

Feature type: polygon

Spatial reference: edit -> import -> movies_geo_spnad83_meters.shp

(This will make your file the same as the movies_geo_spnad83_meters.shp, which is NAD_1983_StatePlane_Massachuesetts Mainland, units = meters)

4. Click ok

5. Close ArcCatalog

 

Edit the shapefile in ArcMap

In ArcMap

1.      Add the high resolution aerial photo file f:\usertemp\arcgis2\BackBay.tif (you will use this as a background layer to trace a building)

2.      Add the empty shapefile “site.shp” to your ArcMap project

3.      Right click in the toolbar area of ArcMap and turn on the editor toolbar

4.      Click editor -> start editing

5.      Choose the F:\usertemp\arcgis2\ folder to edit in

6.      Choose a site in Back Bay to build your new building

7.      Start by selecting the pencil on the editor toolbar, have task set to ‘create new feature’, have target set to ‘site’

Each click creates a new vertex until you double click to close the polygon

Right click and explore the options that appear such as setting a specific distance, angle or location

8.      Modify your building by clicking on the black arrow to the left of the pencil, change your task to modify tasks -> modify feature

this will enable you to do a variety of things including:

move each vertex individually

add or delete vertices

move shapes around

copy and paste shapes

9.      Once you have created your new site you will want to update your attribute table and add a name for it and the number of stories so it can be modeled in 3D. 

10.  Open the attribute table for site.shp

11.  Note the options -> add field is greyed out.  This is because you are in an edit session.

12.  On the editor toolbar click editor -> stop editing and save your edits

13.  In your attribute table click options -> add field

14.  Name your field ‘name’ and choose type: text

15.  Create another field with the name bldg_hgt and type: integer

16.  Start an edit session again for site.shp from the editor toolbar

17.  In the field name, type Educational Theater (ET)

18.  In the field bldg_hgt type 175 (The tallest part of the prudential center lists with a height of 157 feet – this way your site will be taller than the prudential)

19.  Stop editing your file and save your edits

20.  Save your arcmap document and close it.

 

Create a 3D model of Back Bay

From the Start Menu open: All Programs -> ArcGIS -> ArcScene

Add bldgs_bb, BackBay.tif, bare237898.tif, and your new site.shp

Bare237898.tif is a high resolution (0.5 meter) digital elevation model (DEM), which was derived from LIDAR data.  You can use this file for the base layer for your model, but it is larger in size and could slow things down.  It is a good idea to have the computer create a pyramid layer for this file due to its larger size.  You can add the dem to your scene, then make it not display, but still use it as a base layer so things will refresh more quickly.

 

Extrude the buildings and your site so the tall buildings are distinguishable from the short ones.

  1. Right click on the bldgs_bb and open it’s properties
  2. choose the extrusion tab
  3. click “Extrude features in layer”
  4. click ok
  5. click ok

 

 

  1. Next extrude your site.shp by the bldg_hgt column.
  2. navigate the scene to identify buildings you are familiar with, such as the Prudential and John Hancock buildings

 

Drape the buildings and aerial photo over the DEM

  1. Right click the bldgs_bb.shp -> select properties
  2. On the “base heights” tab click the radio button to “obtain base heights for layer from surface: F:\usertemp\arcgis2\bare237898.tif”

 

 

3. Repeat the process for setting the base heights for your site, and the BackBay.tif image.

 

Changing the vertical exaggeration

  1. In the main ArcScene menu select View-> Scene properties
  2. Change the vertical exaggeration of the scene from “none” to “calculate from extent”
  3. Click ok and look at the changes
  4. Repeat this step and this time use a vertical exaggeration 10 times the amount automatically calculated and notice the difference

 

Additional notes:

You may have noticed the BackBay.tif image looks much more grainy than it did in ArcMap.  The default rendering in ArcScene is lower so things can draw more quickly.  You can adjust the rendering

Right click BackBay.tif -> properties

Choose the rendering tab

You will find options for visibility, effects and optimization on this tab

The option to make your layers partially transparent is on the display tab.

ArcScene has animation tools and lets you create movie fly throughs.  Be aware movie files can grow quickly in file size.

Note: A variety of things will take up more memory and make ArcScene work more slowly.  A few common examples include: making the ArcScene display larger, using large files, changing the rendering to make the picture look more clear.